09:30-

10:30

Opening and Keynote

Opening by Vera Toepoel, President of RC33

Keynote Presentation: Katrin Weller

Towards Transparent Computational Social Science – Can Error Frameworks Improve Research with Digital Behavioral Data?

10:45-

12:00

Session 1: Harmonization of Background Variables for Cross- national Surveys

Session chair: Elvira Scholz, Pei-shan Liao

Presentation 1: The International Standard Level of Education [ISLED] for ISCED-2011

Harry BG Ganzeboom, Heike Schröder

Presentation 2: Harmonisation of Background Variables for East Asian Social Survey

Noriko Iwai, Tetsuo Mo

Presentation 3: Investigating the structure of political trust to national institutions: Evidence from the 2016 European Social Survey for Southern Europe

Anastasia Charalampi, Eva Tsouparopoulou, Joanna Tsiganou, Catherine Michalopoulou

Presentation 4: Background Variables in the ISSP

Evi Scholz, Petra Brien

12:30-

13:45

Session 2: Survey Solutions and Hard to Measure Aspects

Session chair: Markus Quandt

Presentation 1: Using comparative survey data to describe the ‘ethnic’ composition of populations – more pitfalls or more opportunities?

Markus Quandt, Mónica Méndez Lago

Presentation 2: Two competing approaches to measuring ethnic origin: ancestry and country of birth

Silke Schneider, Anthony F. Heath

Presentation 3: A measurement evaluation of a six item measure of quality of life (CASP6) across different modes of data collection in the 1958 National Child Development Survey (NCDS) Age 55 years.

Dick Wiggins, George Ploubidis, Matt Brown

Presentation 4:Is it Origin, Destination or Mobility? A Monte Carlo Simulation of the Diagonal Reference Model.

Alessandro Procopio, Robin Samuel

14:00-

15:15

Session 3: The use of passive measurement to replace or augment survey questions

Session chair: Vera Toepoel

Presentation 1: When survey science met online tracking: presenting an error framework for metered data

Oriol J. Bosch, Melanie Revilla

Presentation 2: Smartphone sensor measurements in general population studies

Bella Struminskaya, Peter Lugtig, Barry Schouten

Presentation 3: Analyzing smartphone data for social research

Sonja Malich, Sebastian Bähr, Georg-Christoph Haas, Florian Keusch, Frauke Kreuter, Mark Trappmann

Presentation 4: Consent to data linkage: A meta-analysis

Anne Elevelt, Vera Toepoel, Peter Lugtig

15:30-

16:45

Session 4: Web Probing

Session chair: Katharina Meitinger, Vera Toepoel

Presentation 1: How does the scale affect subjective health ratings? Insights from Web probing

Katharina Meitinger, Cornelia Neuert, Dorothée Behr

Presentation 2: Closed-ended vs open-ended: Evaluating the potential of targeted embedded probes

Cornelia Neuert, Katharina Meitinger, Dorothée Behr

Presentation 3: Response Behavior in Web Probing – The Impact of Open-Ended Probing Questions on Online Survey Behavior

Patricia Hadler

Presentation 4: Integrating web probing qualitative evidence with quantitative data for extending question-and-answer models in cross-cultural survey research

Dörte Naber, Jose-Luis Padilla

09:30-

10:15

Session 5: Methods of Social Network Analysis

Session chair: Claire Wagner, Inga Gaisauskaite

Presentation 1: Dynamics of group-based meetings amongst teenagers

Marion Hoffman, Timothée Chabot, Tom A.B. Snijders

Presentation 2: Interactive network graphs online to analyze surveys

Modesto Escobar

10:30-

12:00

Session 6: Assessing the Quality of Survey Data II

Session chair: Susanne Vogl

Presentation 1: Implementing empirical results of complex models for panel data (dynamic panel models, growth curve models) into dynamic microsimulation Petra Stein, Dawid Bekalarczyk, Eva Depenbrock

Presentation 2: Propensity Score Analysis of the Mixed-mode Effect of a Panel Survey

Su-hao Tu, Ruoh-rong Yu, Tsung-wei Hung

Presentation 3: Systematic non-response socio-demographic differences in time-diary data collection: Evidence from the UK Millennium Cohort Study.

Elena Mylona

Presentation 4: Survey Participation to the First Wave of ITA.LI, the Italian Household Longitudinal Study

Chiara Respi, Emanuela Sala

Presentation 5: What does it mean to be an interviewer? Exploring fieldwork experiences of survey interviewers in Lithuania

Inga Gaižauskaitė, Svajonė Mikėnė, Giedrė Plepytė-Davidavičienė.

12:30-

13:30

Session 7: Assessing the Quality of Survey Data I

Session chair: Iasonas Lampriou

Presentation 1: How Do Survey Mode, Frame, and Fieldwork Effort Affect Data Quality? A Meta-analysis Based on the Data from ISSP, CSES, ESS, and LAPOP Survey Programs.

Adam Rybak

Presentation 2: Coverage error in web surveys in Europe

Alessandra Gaia, Emanuela Sala, Chiara Respi

Presentation 3: The cross-country validation of the WHO-5 well-being index with item response theory and the alignment procedure

Philipp Sischka

13:45-

15:15

Session 8: Research emotions and sensitive topics

Session chair: Martin Weichbold

Presentation 1: Does the sensitiveness of items show in electrodermal activity?

Kathrin Gärtner, Severin Maurer, Martin Weichbold

Presentation 2: Error of Measurement: Researching immigrant populations

Marvin Brinkmann

Presentation 3: Instrumentalization of Emotion During the 2016 U.S. Presidential Election – A Neopragmatist Analysis of the Presidential Nominees’ Media Communication

Sheena F. Bartscherer

Presentation 4: Between Baby and Blood: Interpreting Accounts of Pregnancy Loss on You-Tube

Julia Böcker

Presentation 5: Core Relations Themes as a method for reconstructing emotion from text

Christian von Scheve

15:30-

16:45

Session 9: Innovations and challenges in qualitative research

Session chair: Inga Gaisauskaite, Claire Wagner

Presentation 1: Quality Criteria for Online Qualitative Research

Marlene Schuster, Melanie Hense

Presentation 2: Integrating Q Methodology within In-depth Interviews and Focus Groups: Making Sense of Europeanization Discourses in Georgia

Lia Tsuladze

Presentation 3: Challenges in conducting longitudinal qualitative research among parentally bereaved African children

Mienke van der Brug

Presentation 4: Interviewing adolescents through time: balancing continuity and flexibility in a qualitative longitudinal study

Susanne Vogl

17:00-

18:15

Board Meeting

09:00-

10:15

Session 10: Non-binary sex and gender in general population surveys: Current developments

Session chair: Verena Ortmans

Presentation 1: Beyond the gender item – constructing a gender inclusive questionnaire on sexual health

Kathrin Gärtner, Irina Igerc

Presentation 2: Gender identity and sex: contexts of data collection

Eleanor Scott-Allen, Andrew Nash

Presentation 3: Perception and acceptance of a third gender category in Germany _ Results of a web probing experiment

Verena Ortmanns, Patricia Hadler, Cornelia Neuert

Presentation 4: Gender role attitudes in the Philippines and elsewhere

Harry BG Ganzeboom, Linda Luz Guerrero, Iremae Labucay, Gerardo Sandoval

10:30-

11:45

Session 11: (a) Trust in science / (b) Framing and Taboo in research

Session chair: Inga Gaisauskaite

Presentation 1: The great unmentionable”: the rhetorical construction of taboo on

death and is origins.

Irina Romanova

Presentation 2: Environmental compliance management of production companies – Disclosure identified through methods of observation and scraping of webpages Karsten Boye Rasmussen , Heiko Henning Thimm

Presentation 3: Is science at risk? The bad influence of politicization and industrialization for scientific rigor

Ana Muñoz van den Eynde

Presentation 4: Research partnerships between Indigenous and non-Indigenous researchers

Rosalind Edwards

12:00-

13:30

Session 12: Natural Language Processing: a New Tool in the Methodological Tool-Box of Sociology

Session chair: Renata Nemeth, Ildikó Barna

Presentation 1: An analysis on the framing of Trianon and the Holocaust in Hungary using Natural Language Processing

Árpád Knap, Ildikó Barna

Presentation 2: Understanding online discursive framing of depression via topic models

Renáta Németh, Domonkos Sik, Eszter Katona

Presentation 3: Uncovering the discursive construction of students in higher education

Rakovics Márton , Réka Tamássy

Presentation 4: Detecting Latent Changepoints with Topic Model

Koltai Júlia Anna, Anna Molnár

Presentation 5: Gender disparity in an online video game live-streaming community

Tóbiás Dániel

13:45-

15:00

Session 13:Teaching qualitative, quantitative and mixed methods in social science programs

Session chair: Inga Gaisauskaite, Claire Wagner, Florian Berens

Presentation 1: Teaching qualitative, quantitative and mixed methods in social science programs

Florian Berens

Presentation 2: Applying research methods using multidisciplinary project-based learning

Claire Wagner, Jacques du Toit

Presentation 3: Mapping the terrain: Using instructional technology for teaching research methods

Claire Wagner, Barbara Kawulich, Mark Garner

Presentation 4: Flipping service learning by implementing it in research seminars on empirical methods

Florian Berens

15:15-

17:00

Session 14: General Population Surveys on the Web: Designs

and Usability of Data. Insights from the European Values

Study

Session chair: Michèle Ernst Stähli, Tobias Gummer, Gudbjorg Andrea Jonsdottir, Vera Lomazzi, Ruud Luijkx, Michael Ochsner, Alexandre Pollien

Presentation 1: Identifying and addressing representation issues of different survey designs: Evidence from an experiment with the European Values Study 2017

Jessica Herzing, Michael Ochsner, Alexandre Pollien

Presentation 2: How and when to use multiple imputation when analysing data from a cross-cultural survey in matrix design or with high amounts of missing data

Michael Ochsner, Jessica Herzing, Tobias Gummer

Presentation 3: Measurement equivalence across different designs of the European Values Study 2017

Vera Lomazzi, Daniel Seddig

Presentation 4: Potential consequences of splitting a questionnaire: order effects,

battery splitting and the respondents’ mood

Alexandre Pollien, Oliver Lipps, Jimena Sobrina Piazza

Presentation 5: Effects of mixing modes in general population surveys: Insights from the European Values Study 2017

Tobias Gummer, Pablo Christmann, Christof Wolf

Presentation 6: Measurement invariance in comparative research: an example based on the European Values Survey data

Clelia Cascella

09:15-

10:15

Awards

Session chair: Henrik Andersen

Best Paper by a Young Scholar: Participating in a panel survey changes respondents’

labour market behaviour. Authors: Ruben L. Bach and Stephanie Eckman

Best Paper: Spatial Regression Models: A Systematic Comparison of Different Model Specifications Using Monte Carlo Experiments. Author: Tobias Rüttenauer.

10:30-

12:00

Session 15:Issues in longitudinal data, panel data and secundary data

Session chair: Biagio Aragona

Presentation 1: Scaling Up Cross-National Surveys to a Global Societal Observatory –

the ONBound Data Cumulation and Linkage Project

Markus Quandt, Insa Bechert, Antonia May, Katharina Werhan, Annette Schnabel

Presentation 2: Economic status and elections: A case study on voting behavior and economic hardship

in Rome

Gabriella D’Ambrosio, Giovanni Brancato, Marco Palmieri

Presentation 3: International students in higher education: the effect of student employment on academic performance

Theresa Thies

Presentation 4: Using address data enriched with additional information at regional level to identify less educated respondents before data collection

Uta Landrock, Christian Aßmann

Presentation 5: The Shadow of the Financial Crisis – Macro-level Socio-Economic Development and Feelings of Insecurity in Europe. A Between-Within Analysis of the European Social Survey, 2002-201

Lisa Natter, Dietrich Oberwittler

12:15-

13:45

Session 16: Special session Societal Complexity and Simulation

Session chair: Cor van Dijkum & Dorien DeTombe

Presentation 1: Corona crisis as a complex societal problem

Cor van Dijkum

Presentation 2: Coronavirus: a complex societal problem

Dorien DeTombe

Presentation 3: A simulation of swing voters’ dynamics

Luciano Gallón

Presentation 4: SIMULATION OF COMPLEX SOCIETAL SYSTEMS. PROBLEMS, MODELS AND DATA SET REQUIREMENT

Makarenko Alexander

Presentation 5: Elite familiarity, class and political engagement: The case of Norway

Jan Fredrik Hovden

14:00-

15:15

Session 17: Triangulation and Mixed Methods

Session chair: Susanne Vogl

Presentation 1: Assimilating interpretative and structural data: Merged Methods as an integrative approach of data collection

Wander van der Vaart

Presentation 2: Investigating Social Researchers‘ Methodological Beliefs with

standardized Surveys. Results from Cognitive and Statistical Pretests

Felix Knappertsbusch

Presentation 3: Using mixed methods approaches in validating psychometric instruments

Florian Berens, Kelly Findley

Presentation 4: Mixed Methods Research Citations Network: Communities of Scholars and the Production/Dissemination of Academic Knowledge

Noemi Novello, Alessandra Decataldo

15:30-

17:00

Session 18: Special issues on data collection in survey research

Session chair: Tobias Gummer

Presentation 1: Using fictitious issues to investigate cognitive processes in surveys

Henrik Andersen, Jochen Mayerl, Felix Wolter, Justus Junkermann

Presentation 2: Capturing the interaction between question order effects and visual layout: results from an online experiment

Ádám Stefkovics, Zoltán Kmetty, Koltai Júlia Anna

Presentation 3: Possibilities and Limits of the Joint Modeling Approach for Biosocial Empirical Analysis

Alessandro Procopio, Robin Samuel

Presentation 4: Response behavior and quality of survey data: Comparing elderly respondents in institutions and private households

Jan-Lucas Schanze

Presentation 5: Integrating Probability and Nonprobability Samples for Survey Inference

Joseph W. Sakshaug, Arkadiusz Wiśniowski

17:15-

18:45

Session 19: Challenges in Establishing Validity of Measurements in the Age of Digitalization and Globalization

Session chair: Natalja Menold, Robert Neumann, Peter Graeff

Presentation 1: How Do Reverse-Keyed Items in Inventories Affect Measurement Quality and Information Processing

Natalja Menold

Presentation 2: Cybercrime measurement and issues of validity: Comparing process- generated and survey data in Europe

Julia Kleinewiese

Presentation 3: Are you concerned yet? Measuring privacy concerns

Oliver Brust, Robert Neumann, Hagen von Hermanni

Presentation 4: What determines students’ willingness to share data? Evidence from a Factorial Survey Experiment

Edgar Treischl, Tobias Wolbring

Presentation 5: Are measures of democracy valid in all circumstances?

Claire Durand et al.

18:45-

19:00

Closing Remarks

09:30-

10:30

Opening and Keynote

Opening by Vera Toepoel, President of RC33

Keynote Presentation: Katrin Weller

Towards Transparent Computational Social Science – Can Error Frameworks Improve Research with Digital Behavioral Data?

Computational Social Science addresses societal phenomena using new types of data and computational methods for analyzing them. Among the variety of research approaches, working with data collected from online platforms is particularly popular, as these promise novel insights into their users’ attitudes, behavior or characteristics. But these so-called digital behavioral data all come with their own kinds of challenges, impacting the entire research process – from the conceptional research design to practical issues during data collection and analysis. Research communities are increasingly recognizing these challenges, but often there still is a lack of practically implemented solutions. On first step towards practical solutions could be new structured ways to check research designs for potential sources of challenges (or more specifically: errors) throughout different research phases. By now different “error frameworks” exist that should enable structured discussions about research design choices and may help to better document research based on digital behavioral data. This presentation will illustrate how error frameworks may contribute to more transparent research processes

– and reflect on which additional measures might be needed in the future.

10:45-

12:00

Session 1: Harmonization of Background Variables for Cross- national Surveys

Session chair: Elvira Scholz, Pei-shan Liao

Presentation 1: The International Standard Level of Education [ISLED] for ISCED-2011

Presentation 1 authors: Harry BG Ganzeboom, Heike Schröder

Gradually, the International Standard Classification of Education 2011 is becoming the harmonization instrument in cross-national surveys, such as ESS, EVS and ISSP. ISCED- 2011 brings two innovations relative to its predecessor ISCED97: (A) it distinguishes more main categories (9 versus 7), in particular by adapting distinctions in tertiary education to the BA/MA structure, (B) by introducing a – – somewhat complicated – extension with two further digits. Altogether, ISCED-2011 offers comparative researchers 28 categories to classify country-specific educational qualifications using a

common framework, in much the same way as ISCO works for occupations. In this paper

using ESS R5-R9 data we (re)develop the International Standard Level of Education [ISLED] as a unidimensional hierarchical scaling of ISCED-2011 categories, using a methodology previously applied by Schröder & Ganzeboom (2014) for country-specific qualifications in ESS R1-R4. Having an optimal scale score for ISCED categories is not only convenient for researchers who want to exploit country-specific variations without recurring to crude harmonization, but also allows us to provide precise estimates about the loss of information that researchers encounter when they use crude harmonization or duration as a comparative measure of level of education. We derive these estimates in a latent-variable measurement model that combines optimally scaled qualifications and duration as alternative measurements of the true level of education. Our results, obtained with fresh data from ESS, EVS and ISSP, indicate that ISLED is a very good, but not perfect measure. However, the loss of information with ISLED (3%-5%) is substantively smaller than when one uses crude harmonization or duration of education (10%-15%).

Presentation 2: Harmonisation of Background Variables for East Asian Social Survey

Presentation 2 authors: Noriko Iwai, Tetsuo Mo Presentation 2 abstract:

The purpose of this paper is to discuss the harmonization of background variables for East Asian Social Survey. We specifically focus on education, occupation, income and religion. EASS is a biennial social survey project, which is dedicated to the promotion of comparative studies on diverse aspects of social lives in East Asia. Launched in 2003, EASS focuses on questions and issues that are commonly relevant to four East Asian societies: China, Japan, Korea and Taiwan. EASS modules are incorporated into a GSS- type pre-existing survey framework in each society, rather than creating a new independent cross-national survey. Five modules have been made so far (Family, Culture and Globalization, Health, Social Network and Social Capital and Work and Society). EASS teams set standard background variables for EASS referring to the ISSP SBV. Most of them are straightforward and data are collected through input harmonization:

respondent’s marital status, working hours, household size etc. On the other hand, occupation and educational background variables are collected through output harmonization due to the differences in educational system and occupational classification among four societies. As ISSP, each team computes years of schooling based on respondent’s highest education level/degree and also recodes each society’s occupational classification codes into ILO’s four digit International Standard Classification of Occupations. Years of schooling appears simple, but it does not catch years at vocational school properly at least in Japan. Differences in the value of currency is an issue in dealing with income. EASS allows each team to keep their own way of measuring income: China (annual earnings in actual count), Japan (annual earnings in 19 categories), Korea (average monthly earnings in 21 categories), Taiwan (average monthly earnings in 23 categories). In addition to the absolute income, EASS includes a relative household income question. Harmonisation of respondent’s religion will be also discussed.

Presentation 3: Investigating the structure of political trust to national institutions: Evidence from the 2016 European Social Survey for Southern Europe

Presentation 3 author: Anastasia Charalampi, Eva Tsouparopoulou, Joanna Tsiganou, Catherine Michalopoulou

Presentation 3 abstract:

Political trust to institutions is considered as an important indicator of political legitimacy and the stability of democratic political systems. In the European Social

Survey (ESS) core questionnaire, five pseudo-interval items are used for the measurement of political trust to the following national institutions: parliament, legal system, police, politicians and political parties. The purpose of this study is to investigate the structure and assess the psychometric properties of the political trust to national institutions included in Round 8 (2016) of the ESS for Southern Europe: Italy, Portugal and Spain.

First, the sample of each country was split randomly into two halves. Exploratory factor analysis (EFA) was performed on the first half-samples in order to assess the construct validity of the construct. In all cases, EFA resulted in a one-factor solution based on all five items. Then, the structure was validated by carrying out Confirmatory factor analysis (CFA) on the second half-samples. CFA performed on the full samples resulted in an adequate, an inadequate and an acceptable model fit for Italy, Portugal and Spain, respectively. Based on the full sample, the psychometric properties of the resulting unidimensional scales were assessed.

The analysis produced three unidimensional scales that were both reliable and valid. This work could be extended to cover all participating countries of this and any other Round of the ESS.

Presentation 4: Background Variables in the ISSP Presentation 4 authors: Evi Scholz, Petra Brien Presentation 4 abstract:

The International Social Survey Programme (ISSP) is a cross-national social science survey programme with members from around the globe who voluntarily collaborate based on a common agreement and on working principles.

In contrast to other cross-national programmes with hierarchical structures, the ISSP is a democratic project of equal members. This particularity results in country-specific elements (for example, national survey fielding, data management and documentation) on the one side and central elements (for example, study monitoring of national data collection, and control of national data on compliance with ISSP requirements) on the other side.

An ISSP questionnaire consists of two parts: the source questionnaire on the thematic module and the obligatory ISSP background variables (BV). While the source questionnaire with an input harmonisation approach is developed by all ISSP members in a joint effort, the ISSP BV result from a mixed mode approach: The ISSP BV are implemented by individual ISSP members combining national needs of asking country- specific questions and complying with ISSP measurement requirements described in the ISSP BV guidelines (measurement goal, coding frame, filter and routing conditions). The construction of the harmonised ISSP BV from the national BV is documented by ISSP members in an obligatory report as part of the data delivery.

The presentation describes the concept of the ISSP BV in general, elaborates on the way the ISSP inspects the data in order to guarantee the quality and gives an example for these monitoring tasks.

12:30-

13:45

Session 2: Survey Solutions and Hard to Measure Aspects

Session chair: Markus Quandt

Presentation 1: Using comparative survey data to describe the ‘ethnic’ composition of populations – more pitfalls or more opportunities?

Presentation 1 authors: Markus Quandt, Mónica Méndez Lago Presentation 1 abstract:

At least since Alesina et al.’s (2003) work on the effect of “ethnic fractionalization” on economic growth, measures of ethnic heterogeneity figure prominently in analyses not only of the economic performance of societies, but in relation to solidarity, social capital, internal and external conflict etc.However, little is known about the reliability and validity of the source data – proportions of particular ethnicities within each population – used for the computation of a fractionalization index. Often, these data are taken from sources as diverse as statistical offices’ reports, expert estimates, and ‘meta searches’ over research publications. There are obvious problems with the comparability and reliability of such sources.Given the growing wealth of data from comparative surveys across the world, it therefore seems tempting to draw on the information that such surveys collect on the ethnicity of their respondents, often as part of their standard demographics. The implied assumption is that most of these surveys follow a systematic approach towards achieving comparability, which should ameliorate problems stemming from the ad hoc nature of the former method of data compilation. But again, there are limitations. Besides the obvious one, – that sample sizes might be too small to derive reliable estimates for ethnic groups with small population shares -, there likely still is methodological and conceptual variation that is not always obvious. This includes differences in target populations (all residents vs citizens only); differences in data collection practices (e.g. the translation of questionnaires to minority languages), and others.To assess the balance of pitfalls and opportunities of a survey-based approach, the talk will present explorative analyses of ethnic group shares estimated from different comparative survey programs covering European countries (such as the ESS, ISSP, EVS,

IntUne, …), and compare those to the available official statistics and to the original

Alesina data.

Presentation 2: Two competing approaches to measuring ethnic origin: ancestry and country of birth

Presentation 2 authors: Silke Schneider, Anthony F. Heath Presentation 2 abstract:

This paper compares measures of ethnic origin based on respondents’ reported ancestry and on respondents’ reports of their own and their parents’ countries of birth. Both kinds of measures have been implemented in parallel in the European Social Survey (ESS) since 2014. We pool the data from two waves of the ESS, and we code origins according to the same detailed framework (the European Standard Classification of Cultural and Ethnic Groups). Our analysis will then be conducted in three steps: firstly, the concept of ethnic origin is defined and distinguished from other measures of ethnicity such as ethnic identity and migration background. Secondly, the two different empirical indicators of ethnic origin available in the ESS are contrasted, namely

respondents’ (self-reported) ancestry and respondents’ and parents’ country of birth; we consider how well these two measures reflect the theoretical concept, how they were implemented in the ESS, and what can be done with the resulting data. Thirdly, we present results from two empirical analyses, a) investigating the relationship between the two indicators to check their convergent validity and b) comparing their usage in multivariate regression models to see whether they differ in predictive validity. To conclude the presentation, we will summarize the results and discuss the strengths and weaknesses of both indicators, such as their predictive validity, their ability (or otherwise) to identify ethnic minorities beyond the second generation, and their ability to distinguish autochthonous from allochthonous migrants and to identify sub-national groups.

Presentation 3: A measurement evaluation of a six item measure of quality of life (CASP6) across different modes of data collection in the 1958 National Child Development Survey (NCDS) Age 55 years.

Presentation 3 authors: Dick Wiggins, George Ploubidis, Matt Brown Presentation 3 abstract:

The purpose of this paper is to evaluate the performance of a six-item self-report (CASP6) as developed from a longer 12-item version of a well-established measure of quality of life (www.casp19.com). The focus of the analytical assessment is an examination of the impact of the mode of data collection on the measurement

properties of CASP6 in the wider context of an evaluation of the ‘sequential mixed- mode’ design adopted for the UK’s 1958 National Child Development Study Age 55 Survey. Cohort members were first invited to complete the survey online, and then by telephone if they had not completed the online survey after 5 weeks. A general – specific measurement model captures both the unidimensionality of the scale and the nature of the wording of items across modes (CFI=0.993, TLI=0.984 & RMSEA=0.068, N= 9023).

Similar assessments for either online or telephone alone and mixed mode with a telephone option revealed confirmatory results for the use of CASP6 as a standalone measure of quality of life.

Presentation 4: Is it Origin, Destination or Mobility? A Monte Carlo Simulation of the Diagonal Reference Model.

Presentation 4 authors: Alessandro Procopio, Robin Samuel Presentation 4 abstract:

There is a long-standing interest in the net effects of socioeconomic origin, destination and mobility on different facets of, e.g., attitudes, behaviors and intergenerational transmission of wealth. Among applied social scientists, Sobel’s Diagonal Reference Model (DRM) is currently considered the most suitable tool to disentangle origin, destination, and mobility effects. However, several mixed or even null evidences contradicting theoretical predictions, and the lack of literature on the behavior of the model under various conditions render the situation puzzling. We report on a Monte Carlo Simulation of DRM, considering three scenarios: a) linear dependency, b) selection into unobservables and c) reverse causality. We employ a theory-driven data generation process, in order to get reasonably realistic and substantial data. Our preliminary results suggest little to no bias in the estimation of net mobility effects. With a view to enable causal inference in further applications, more research should, however, address the implementation of the DRM to longitudinal data.

14:00-

15:15

Session 3: The use of passive measurement to replace or augment survey questions

Session chair: Vera Toepoel

Presentation 1: When survey science met online tracking: presenting an error framework for metered data

Presentation 1 authors: Oriol J. Bosch, Melanie Revilla Presentation 1 abstract:

Metered data (also called “web log data”, “digital trace data”, “web-tracking data” or “passive data”) is a type of Big Data obtained from a meter willingly installed or configured by participants on their devices (PCs, tablets and/or smartphones). A meter refers to a heterogeneous group of tracking technologies that allow sharing with the researchers, at least, information about the URLs of the web pages visited by the

participants. Depending on the technology used, HTML, time or device information can also be collected. Metered data is objective, free of human memory limitations and produced in real-time. Therefore, metered data has a great potential to replace part of survey data or to be combined with survey data to obtain a more complete picture of reality. Metered data, nevertheless, needs to be used properly. It is crucial to understand its limitations to mitigate potential errors. To date, some research has explored potential error causes of metered data. However, a systematic categorization and conceptualization of these errors is missing. Therefore, in this presentation, we present a framework of all errors that can occur when using metered data. We adapt the Total Survey Error (TSE) framework (Groves et al., 2009) to accommodate it to the specific error generating processes and error causes of metered data. The adapted error framework shows how the unique characteristics of metered data can affect data quality, but also allows comparing metered data errors with survey errors, since it is based on the TSE framework. Hence, the adapted framework can be useful when using metered data alone or in combination with surveys, to choose the best design options for metered data, but also to make better informed decisions while planning when and how to supplement or replace survey data with metered data.

Presentation 2: Smartphone sensor measurements in general population studies Presentation 2 authors: Bella Struminskaya, Peter Lugtig, Barry Schouten Presentation 2 abstract:

Collecting data via smartphone sensors can provide detailed information about behavior, potentially reduce respondent burden and improve measurement accuracy. However, respondents need to be willing and able to collect such data. Otherwise the results can be biased. We focus on the mechanisms of willingness and compliance with requests to collect smartphone sensor measurements utilizing data from two general population Dutch surveys (LISS Panel and Statistics Netherlands Survey). To investigate mechanisms of wording, privacy concerns, and control of data collection, we randomly assigned smartphone users to three conditions (benefit vs. neutral framing, assurance of confidentiality, ability to revoke measurements) and asked about willingness to share GPS location, take photos and videos and then to collect these measurements.

Nonwilling respondents were asked to provide reasons for nonwillingness. We use rich administrative data linked to the survey data to estimate nonconsent bias. We find that willingness varies across sensor types and tasks. Willingness and consent are influenced by study sponsor and ability to control data collection, but not by wording of the request. Higher trust, lower privacy concerns, more experience with smartphone sensors, and survey experience are associated with higher consent. Nonconsent bias is generally larger than nonresponse bias, but in some cases biases counteract.

Presentation 3: Analyzing smartphone data for social research

Presentation 3 authors: Sonja Malich, Sebastian Bähr, Georg-Christoph Haas, Florian Keusch, Frauke Kreuter, Mark Trappmann

Presentation 3 abstract:

Compared to data from traditional social science methods, smartphone data share many characteristics of Big Data. Smartphone data comprise a variety of data formats (e.g., geo-positions, movement patterns, communication histories, app usage) that are collected at high frequency. In addition, passive measurement keeps the burden on study participants low and reduces potential bias due to inaccurate information or recall bias as known from self-reports in surveys. Smartphone data thus provide rich information about social behaviors over the course of a day at a very fine-grained level.

The aim of this presentation is to show how these data can be used to answer specific

social science research questions and how they can overcome limitations of classical survey methods. To this end, we use data from the IAB-SMART study in which a smartphone app was used to collect survey and sensor data from 623 members of a representative panel study in Germany over a period of six months. The potential of these data is further enhanced by the special study design through which the smartphone data can be linked to data from the IAB’s Labour Market and Social Security (PASS) Panel as well as to individual administrative data on participant’s employment biographies such as wages. We use these data to show how the novel measurement (e.g., of step counts, speed of movement, location, smartphone use) can be used to display daily trajectories, identify everyday structures, and compare behaviors of different social groups.

Presentation 4: Consent to data linkage: A meta-analysis Presentation 4 authors: Anne Elevelt, Vera Toepoel, Peter Lugtig Presentation 4 abstract:

Linking survey data to all kind of other data sources becomes increasingly important and the widespread adoption of digital technologies is even further expanding opportunities for survey researchers to enhance survey research. This can include all kind of data, but most researchers currently use sensor data (e.g. location, accelerometer), biomarker data (e.g. blood, saliva), social media data (e.g. Facebook, Twitter) or administrative records (e.g. employment data, health records, social security records).There are several advantages of linking data, but the main reasons are the decrease of respondent burden (by reducing the number of questions) and improving data quality (by collecting more accurate and detailed data than respondents can provide). However, a crucial element of being able to do research successfully with linked datasets is to get respondents to consent in order to prevent non-consent bias. Ineffective ways of asking for consent can lead to low consent-rates, as various aspects of the consent question (wording, sponsor, placement, etc) seem to influence the consent rates. We conducted a meta-analysis, in which the results of multiple experimental studies on consent for data linkage were combined and re-analyzed. The main contribution of this paper is to systematically present the state of the art of asking for consent. Based on the results of this study, guidelines are developed on effective ways to ask for consent to link survey data to other types of data.

15:30-

16:45

Session 4: Web Probing

Session chair: Katharina Meitinger, Vera Toepoel

Presentation 1: How does the scale affect subjective health ratings? Insights from Web probing

Presentation 1 authors: Katharina Meitinger, Cornelia Neuert, Dorothée Behr Presentation 1 abstract:

The self-rated health item is a crucial and frequently used question that is implemented in most large-scale surveys. However, the large scale surveys use different scales for this item. For example, the European Social Survey (ESS) uses a balanced 5-point scale

running from “very good” to “very bad” with “fair” as mid-point of the scale whereas the GESIS Panel opted for “moderately” as mid-point. The Survey of Health, Ageing and Retirement in Europe (SHARE) and the International Survey Program (ISSP) use an unbalanced scale in which positive categories predominate and with “excellent” and “bad” as end points. The question remains how the different response scales influence the answer behavior of respondents and whether different associations are triggered for each scale point of each respective scale. This presentation discusses findings of a split

split-ballot experiment that was implemented in a web survey with a sample of 1,600 German panelists of an online access panel. The study was fielded in July 2019. The experiment compared three different scale versions of the self-rated health question. Each scale version was followed by a closed-ended probe that asked respondents which factors they considered when deciding on an answer category. The aim of the closed- ended probe was to assess whether associations differed by scale point. The closed- ended probes were developed based on insights from previously conducted cognitive interviews in the U.S. and in Germany. In addition to a content perspective, we will assess whether the different scale versions affect response quality (e.g., item non- response) and response behavior (e.g., response time). Findings and implications are discussed.

Presentation 2: Closed-ended vs open-ended: Evaluating the potential of targeted embedded probes

Presentation 2 authors: Cornelia Neuert, Katharina Meitinger, Dorothée Behr Presentation 2 abstract:

The method of web probing integrates cognitive interviewing techniques into online surveys and is increasingly used to evaluate questions by collecting data on respondents’ answer processes. Typically, web probes are administered directly after the question to be tested as open-ended questions with text fields. While the use of open-ended probes in web surveys has shown promising insights in discovering problematic survey items and inequivalence across countries, it is generally acknowledged that open-ended questions are more burdensome to answer for respondents and face more item- nonresponse. A second possibility of administering probing questions is in a closed- ended question format. The response options for the closed-ended questions are developed by relying on the findings of previous collected qualitative cognitive interviewing data. The implementation of closed-ended probes drastically reduces the costs and burden involved in the data processing and analysis stages compared to open- ended questions, because it omits the development of coding schemes, the coding of the responses and reduces the response burden of answering. However, how well both probe types compare in terms of their explanatory power (e.g. the prevalence of

respondents’ associations and error types) is still open. In this paper, we address the Following two research questions: 1) Do closed-ended probes help to assess the validity of survey questions and do they arrive at similar substantive conclusions than open- ended probes? 2) Which approach performs better regarding response quality? In particular, we compare the amount of insights in terms of validity (variety and prevalence of themes) as well as indicators of response quality (e.g. item nonresponse, nonresponse bias, response times). Based on a sample of 1,600 German panelists of an online access panel, we conducted a web experiment comparing the responses of closed-ended vs open-ended probing questions on three questions under consideration. The study was fielded in July 2019. Findings and implementations are discussed.

Presentation 3: Response Behavior in Web Probing – The Impact of Open-Ended Probing Questions on Online Survey Behavior

Presentation 3 author: Patricia Hadler Presentation 3 abstract:

Web probing is the implementation of probing techniques from cognitive interviewing into web surveys. It has emerged as an increasingly popular pretesting method, as it combines the qualitative approach of cognitive interviewing with higher case numbers. This makes cognitive online pretests an attractive opportunity to collect quantitative data, which would usually be analyzed in a standard pretest, such as response

distributions and response latencies. Using such data assumes that respondents

demonstrate comparable response behavior in web probing and the later online survey. However, probing questions require respondents to reflect their previous answers, which in turn may impact how they process subsequent survey questions. Differences could occur both on the level of responses themselves, such as non-response and response distribution, or on the level of response behavior, such as response latency, revisiting survey questions and answer changes. The present research examines whether the implementation of open-ended probing question influences how respondents answer survey questions. In a web survey, six survey pages with single questions or item batteries were fielded. Respondents were randomly assigned to a condition in which (A) an open-ended probe was asked directly after each question/item battery or (B) no open-ended probes were asked. The study was carried out in Germany in autumn of 2020. A total of 2,000 respondents participated, with quotas for gender, age, education and region. A client-side paradata script captured response behavior during the survey. Results indicate that respondent behavior differs significantly between web probing and the later online survey in some, but not all questions and measures. Implications for web probing methodology are discussed.

Presentation 4: Integrating web probing qualitative evidence with quantitative data for extending question-and-answer models in cross-cultural survey research Presentation 4 authors: Dörte Naber, Jose-Luis Padilla

Presentation 4 abstract:

The growing use of web surveys is a challenge for the well-known model of response process introduced by Tourangeau (1984, 2018), and for the common methods of pretesting survey questions. On the other hand, the current development of qualitative methods like “web probing” allow researchers to obtain evidence of the response processes while respondents are answering the web survey questions, and integrate such evidence with quantitative data: item responses, response time, demographics, etc. In our presentation, we will first review briefly current models of the response processes and secondly, analyze and integrate qualitative and quantitative data from a research project aimed to extend the knowledge of pros and cons of web probing. We resort to qualitative evidence from Web Probing (WP) method along with psychometrics for survey data collected of 1,000 participants (500 in Germany and 500 in Spain) to single and multi-items of the 8th European Social Survey Round within a mixed-method design. In this presentation, our main focus will be on the content of three web probes:

category-selection, specific and comprehension probes, and if there is an effect of the probe sequence and demographics of the participants. We will also discuss on how to integrate web probing qualitative evidence with quantitative results from psychometrics

within a mixed research framework for improving cross-national survey data quality.

09:30-

10:15

Session 5: Methods of Social Network Analysis

Session chair: Claire Wagner, Inga Gaisauskaite

Presentation 1: Dynamics of group-based meetings amongst teenagers Presentation 1 authors: Marion Hoffman, Timothée Chabot, Tom A.B. Snijders Presentation 1 abstract:

Many social relations are not only situated at the level of dyads but also within social groups. Typical examples are friendship groups, working groups, or sports teams. These groups are instrumental in shaping individuals relationships and have long-lasting effects on various social outcomes. The composition of these groups is the result of a complex combination of mechanisms at different levels. They depend on the dyadic relationships between their members, but they are also driven by group mechanisms amongst all group members. Moreover, many instances of groups are constrained by exogenous structures, such as institutional or physical restrictions. Although theories on social groups, social foci, social circles, and others are manifold, little is known regarding the mechanisms underlying the formation and evolution of social groups. Importantly, these mechanisms do not necessarily mimic the ones observed for social ties. New statistical models thus need to consider social groups as the outcome of a complex combination of mechanisms on different levels. In this talk, we propose such a model to draw inferences on dyadic and group level mechanisms underlying the evolution of social groups when time-stamped data are available. Specifically, we focus on the case where individuals form non-overlapping groups and represent these groups by a partition of the actor set. The model is applied in an empirical case in which we collected information on sharing tables during meals amongst participants of a summer camp. Data comprises individual attributes and previous acquaintances of 60 teenagers (monadic and dyadic data), as well as the table compositions during the three weeks of the camp (social group data).

We first show how the physical constraints of space in the canteen can be taken into account. We further examine how homophilous preferences, prior social ties, and serendipitous encounters affect the evolution of the social groups.

Presentation 2: Interactive network graphs online to analyze surveys

Presentation 2 author: Modesto Escobar Presentation 2 abstract:

Interactive network graphs online to analyze surveys.

Graphs have been employed not only to solve topographic problems and to represent social structures, but also to show the correlation between variables according to casual models. Path analysis and structural equations models are indeed well known by social scientists, but both were restricted to quantitative variables at their early stages. In this paper, we will propose a new way to display the connections between qualitative variables in a similar way to the correspondence analysis, but using another set of multivariate techniques, such as lineal and logistic regression, mixed with network analysis.

The NCA (NetWork Coincidence Analysis) is to be used for the exploratory analysis of survey data. For this purpose, nodes represent the different categories of the selected variables, while links symbolize the relationships between the different variables. One of the specific uses of this analysis technique involves the characterization by diverse sociodemographic variables of different response profiles. Besides correlation measures,

the proposed representations can be based on solving several equations and selecting only those coefficients with a significant positive relationship. By doing so, graphs are obtained selecting the categories with predicted proportions or means significantly greater than those of the sample.

Furthermore, to increase the analytic power of these tools, they have interactive characteristics online, which include either the selection of the elements according to their size or attributes, and the filter of the most central and strongest links.

The first part of the paper deals with the statistical basis of these representations; the second proposes programs to make them possible, and the third gives examples of their use in international comparative surveys, such as the World Values Survey and the

European Social Survey.

10:30-

12:00

Session 6: Assessing the Quality of Survey Data II

Session chair: Susanne Vogl

Presentation 1: Implementing empirical results of complex models for panel data (dynamic panel models, growth curve models) into dynamic microsimulation Presentation 1 authors: Petra Stein, Dawid Bekalarczyk, Eva Depenbrock Presentation 1 abstract:

Access to economic, social and cultural resources is largely determined by educational attainment and suc- cess on the labor market, especially in Germany. To understand the underlying mechanisms deeply, individ- ual educational and professional pathways have to be analyzed by means of panel data, focusing on endog- enous dynamics within life courses. This can be done by dynamic panel models (Arellano & Bond 1991; Wil- liams et al. 2018). Another suitable approach to model life courses is growth curve modeling (Bollen und Curran 2005). Efforts exist, to combine these two types of models (Bauldry & Bollen 2018, Mund & Nestler 2019). However, both classes of models have by themselves statistical problems that are not fully solved until now (for dynamic panel models: Leszczensky & Wolbring 2019). To make things worse, empirical appli- cations for the combined models are rare, since they were developed recently. We want to contribute to a better understanding of those complex models by applying them to an analysis of educational and professional pathways of third-generation migrants in Germany (project funded by the German Research Foundation). Since members of this generation are young, we additionally want to forecast these pathways by means of a microsimulation (Gilbert und Troitzsch 2005). This brings up the additional challenge to implement the results from the panel data models used here into the microsimulation model. This is especially difficult, when the starting data for the microsimulation model does not coincide with the estimation sample, which is the case here. Some approaches exist for static panel models, to solve these problems (Richiardi 2014). We have tested those approaches and started to develop strategies, to expand them to dynamic panel models and growth curve models (Bekalarczyk & Depenbrock 2020). These strategies will be presented together with empirical results from the project mentioned above.

Presentation 2: Propensity Score Analysis of the Mixed-mode Effect of a Panel Survey

Presentation 2 authors: Su-hao Tu, Ruoh-rong Yu, Tsung-wei Hung Presentation 2 abstract:

In order to reduce survey cost and to reach more respondents, a mixed mode has become widely adopted in panel surveys, especially after a few waves of data collection. However, the respondents are usually self-selected instead of randomly assigned to

survey mode. To distinguish the mixed-mode effect from the sample-selection effect has become crucial. In this paper, we use propensity score analysis, which can reduce the

self-selection bias, to analyze the mixed-mode effect on the response to well-being in a large-scale panel survey. The data are from the Panel Study of Family Dynamics in Taiwan. From its first data collection to the 2016 survey, almost all questionnaires were completed by face-to-face interview. Starting with the 2018 survey, an internet self- administered questionnaire has been adopted along with the face-to-face interview.

Using the data collected in 2018, we adopt one-to-one matching with replacement, Radius matching, and Kernal matching of propensity score analysis commonly used and suggested in the literature. We choose the response to wellbeing including happiness and life satisfaction to examine the mixed-mode effect due to the items are to some degree sensitive to social desirability. The findings are beneficial to our understanding of the mixed-mode effects associated with panel surveys.

Presentation 3: Systematic non-response socio-demographic differences in time-diary data collection: Evidence from the UK Millennium Cohort Study.

Presentation 3 author: Elena Mylona Presentation 3 abstract:

Response rates are often not adequate indicators of non-response bias, as they focus on the percentage of participants not taking part in a study, rather than their shared characteristics as a group. Non-response increases for additional data collection, due to respondent burden, given the extra time and effort required, with existing research showing the effect of certain socio-demographic factors on response rates, such as socio-economic status, busyness and ethnicity. This study used the Millennium Cohort Study (MCS), a UK-representative, longitudinal multi-disciplinary study of children born

between 2000 and 2002, where in the 6th wave (age 14), time use diary data (TUD) were collected alongside the survey questionnaires. Paradata on eligibility were requested from the Center of Longitudinal Studies on the TUD studies and on productive/unproductive cases of the TUD. Multivariable logistic regression was used to model response vs non-response adjusting for complex survey design. Independent variables based on survey data included sex, ethnicity, subjective health status of the cohort member, highest parental educational qualification, number of average working hours to estimate ‘busyness’ of the most educated parent, and family structure.

Consistent with existing literature, findings suggest that the higher the parental education, the higher the odds of the children participating in the additional data collection. Educated parents are more likely to understand the merits of research, and therefore allow and encourage their children to participate. Females, children living in a two-parent household, and children with good subjective health also have higher odds of responding. Those reporting Indian ethnicity had higher odds of responding than the white majority, while Pakistani/Bangladeshi/Black children were less likely to respond. Busyness was not associated with response. Understanding the key demographic differences between study participants and non-participants is paramount in health and social research, as it can help researchers interpret appropriately their studies’ substantive findings.

Presentation 4: Survey Participation to the First Wave of ITA.LI, the Italian Household Longitudinal Study

Presentation 4 authors: Chiara Respi, Emanuela Sala Presentation 4 abstract:

Longitudinal household surveys are important research resources in the social sciences. Obtaining good quality data is particularly challenging with this kind of surveys, because of the specific methodological issues that we face when designing household longitudinal surveys and collecting longitudinal data. Specifically, obtaining high

response rates at Wave 1 as well as maximizing response in the following waves are amongst the main challenges that we face.

Despite of the relevance of these issues, there is relatively little research documenting Wave 1 contact and response process in household longitudinal surveys. The aim of this paper is to document survey participation at Wave 1 of the Italian household longitudinal survey ITA.LI (Italian Lives). First, we will describe the contact process, e.g., outcome of each contact, reasons for non-contact, survey materials left at the household. Second, we will investigate the response process, focussing both on response at the individual and household level, e.g., characteristics of the person who refuses, reasons for refusals, household characteristics. We will investigate whether there are any specific contact or response patterns, that may vary in relation to the area of residence, e.g., regions of Italy, degree of urbanicity. Third, we will explore whether the responding sample is representative of the Italian general population.

Presentation 5: What does it mean to be an interviewer? Exploring fieldwork experiences of survey interviewers in Lithuania

Presentation 5 author: Inga Gaižauskaitė, Svajonė Mikėnė, Giedrė Plepytė-Davidavičienė.

Presentation 5 abstract:

Interviewers play a focal role when a survey is administered in a form of in-person (face- to-face) interviews. Survey literature covers several advantages that interviewer- moderated surveys have regarding data quality. There is more control over data collection process in general, a better fulfilment of a questionnaire, a possibility to administer longer questionnaires or survey complex issues, a possibility to provide support for respondents during an interview (e.g., explain a task or probe for an answer); also, higher response rates of face-to-face surveys may also be linked with the work of interviewers. Next to these advantages, there are also challenges posed by the presence of an interviewer. Interviewers may have diverse effects on respondents’ responses thus causing biases in survey data. Both advantages and challenges indicate that face-to-face surveys rely heavily on the quality of interviewer work as leading towards higher or lower quality of data. There have been many attempts to discuss and introduce preventive measures for potential problems in interviewer work, such as trainings or guidelines for interviewers as well as mechanism of control over their work. However, there is less literature that turns towards experiences of interviewers themselves and highlights their perspective on what they see as key preconditions for success in their work and what are the main challenges they face when administering face-to-face surveys. The paper presents results of a qualitative research conducted in Lithuania. Based on in-depth individual interviews with experienced survey interviewers, we aimed to understand the reality of survey-field work from the perspective of interviewers and how it links to the quality of survey data. In the presentation, we will focus on how interviewers and other involved parties (e.g., respondents, employers) perceive interviewer work; what motivates and de-motivates interviewers; what are the risks that interviewers face in their work as well as what support they receive (or do not receive); what are the key factors that they see as fostering or hindering the quality of their work.

12:30-

13:30

Session 7: Assessing the Quality of Survey Data I

Session chair: Iasonas Lampriou

Presentation 1: How Do Survey Mode, Frame, and Fieldwork Effort Affect Data Quality? A Meta-analysis Based on the Data from ISSP, CSES, ESS, and LAPOP Survey Programs.

Presentation 1 author: Adam Rybak Presentation 1 abstract:

Survey research faces many obstacles, from which nonresponse and measurement biases may be the most important. Methods used to address these issues can be, in many cases, counterproductive. This presentation aims to analyze the effectiveness of these methods – mainly the fieldwork effort and survey mode. Their effect (with the sampling design as a control variable) on the survey errors will be estimated. The analysis will be based on the documentation and datasets from the International Social Survey Programme, Comparative Study of Electoral Systems, European Social Survey, and Latin American Public Opinion Project. These cross-country projects were selected because of their global scale and (especially in the two first cases) plurality of survey methods used. Because there are no straightforward methods of estimating both errors, the internal criterium proposed by Kohler (concerning the proportion of genders in two heterosexual partners households) will be used for the first one – nonresponse bias. For the second, because measurement error consists of many components that cannot be measured at once, the item nonresponse ratio will serve as the indicator of satisficing activities. Created random-effects meta-regression models, based on data from studies conducted in countries from all continents, will estimate the impact of such variables as the number of contacts with the respondent, advance letters, material incentives, survey mode used (or the mix of modes) and sampling design (for control purposes) on quality of sample structure and item nonresponse ratio. The research is funded by the Polish National Science Center, Preludium grant no. 2019/33/N/HS6/00322.

Presentation 2: Coverage error in web surveys in Europe Presentation 2 authors: Alessandra Gaia, Emanuela Sala, Chiara Respi Presentation 2 abstract:

Drawing on the total survey error (TSE) framework, this paper analyses coverage bias in web surveys. Coverage error arises in Internet surveys that exclude the population that lacks the equipment and the skills to access the Internet but nonetheless aim to generalise results to the general population. If the internet population and the non- internet population differ in variables of interest for research purposes, then coverage bias arises. While, in Europe, Internet penetration has increased over time and, especially in areas where it is still low, it may increase further over the next years, researchers have argued that, over time, the non-Internet population might become more and more different from the rest of the population, and this phenomenon might increase coverage bias. Indeed, both the literature on the “digital divide” as well as the survey methodology literature has shown marked socio-demographic differences in Internet use. Applying multilevel analysis to Eurobarometer data (2010-2018) we put forward previous research (Mohorko, de Leeuw, & Hox, 2013) to analyse coverage rate and bias in web surveys in Europe, and monitor its evolution over time. Specifically, we answer the following research questions: what is the Internet coverage rate? Are there socio-economic and demographic differences between individuals having access to the internet and individuals without internet access? Does coverage error in web surveys lead to coverage bias in key social indicators such as political participation, health, and organisation of leisure time? Are there differences across European Countries in the coverage rate and coverage bias in key social indicators? The implication of research findings on survey practice will be discussed.

Presentation 3: The cross-country validation of the WHO-5 well-being index with item response theory and the alignment procedure

Presentation 3 author: Philipp Sischka Presentation 3 abstract:

Background: The five-item World Health Organization Well-Being Index (WHO-5) is a frequently used brief stan- dard measure in large-scale cross-cultural clinical studies. Despite its frequent use, some psychometric questions remain that concern the choice of an adequate item response theory (IRT) model, the evaluation of reliability at important cutoffpoints, and most importantly the assessment of measurement invariance across countries.

Methods: Data from the 6 th European Working Condition survey (2015) were used that collected nationally representative samples of employed and self-employed individuals ( N = 43,469) via computer-aided personal interviews across 35 European countries. An in- depth IRT analysis was conducted for each country, testing dif- ferent IRT assumptions (e.g., unidimensionality), comparing different IRT-models, and calculating reliabilities.

Furthermore, measurement invariance analysis was conducted with the recently proposed alignment procedure.

Results: The graded response model fitted the data best for all countries. Furthermore, IRT assumptions were mostly fulfilled. The WHO-5 showed overall and at critical points high reliability. Measurement invariance anal- ysis revealed metric invariance but discarded scalar invariance across countries. Analysis of the test characteristic curves of the aligned graded response model indicated low levels of differential test functioning at medium levels of the WHO-5, but differential test functioning increased at more extreme levels.

Limitations: The current study has no external criterion (e.g., structured clinical interviews) to assess sensitivity and specificity of the WHO-5 as a depression screening- tool.

Conclusions: The WHO-5 is a psychometrically sound measure. However, large-scale cross-cultural studies should employ a latent variable modeling approach that accounts for non-invariant parameters across countries (e.g., alignment).

13:45-

15:15

Session 8: Research emotions and sensitive topics

Session chair: Martin Weichbold

Presentation 1: Does the sensitiveness of items show in electrodermal activity? Presentation 1 authors: Kathrin Gärtner, Severin Maurer, Martin Weichbold Presentation 1 abstract:

To assess the sensitiveness of survey questions different methods are used: for example, respondents can be asked within the survey or during cognitive test how they feel about the question or item non-response can be analysed. However, to detect sensitive questions in pretests without bothering participants with additional probings we explored a third option by asking: Do selected physiological and behavioural measures differ when answering sensitive (which are assumed to evoke feelings of embarrassment or unpleasantness) as compared to non-sensitive questions? To address this question, we conducted and experiment with 84 undergraduate students, which were confronted with 12 pretested question in a laboratory setting: 9 normal questions and 3 sensitive questions concerning income, bowel movement and masturbation. Results show that whereas sensitive questions do not seem to differ in eye movement and reaction time

there seems to be an effect on electrodermal activity (EDA). During the periods when the sensitive questions were presented a significant higher number of peaks in EDA were

counted. However, there was also one non-sensitive question which was characterized by a higher number of peaks: a question where participants had to estimate the distance to the next supermarket. As this question requires more cognitive effort we concluded that EDA not only indicates sensitive items but also items with a higher mental load.

Possible implications for survey construction and survey research are discussed.

Presentation 2: Error of Measurement: Researching immigrant populations

Presentation 2 author: Marvin Brinkmann Presentation 2 abstract:

Perceived discrimination is nothing to be proud of. Framing and wording experiments on avoiding nonresponse in a sensitive issue.

Discrimination is a salient, yet sensitive issue. Does the saliency also lower the sensitivity and lead to less unit non-response on discrimination experiences in the population? This paper will break this line of argumentation and show that the European Social Survey (ESS) item on discrimination experiences (Billiet 2001) heavily underscores actual discrimination experiences for a long time. This also seems to be true as only 9-13% of people with migrations background in the ESS indicate discrimination experiences in the repeated measure whereas other studies for example in Germany indicate values of up to 80% (Sauer 2018). Evidence comes from four experimental designs in two weighted online surveys on people with migration background, both administered in Germany in December 2020. Among classical wording experiments, a framing experiment was performed to lower potentially problematic answers. While the previous level of the European Social Survey could be replicated without experimental manipulation of the question, hence, excluding a period effect, the manipulated outcomes can more than double the reporting of experienced discrimination among people with migration background. The results in the German context are so clear that similar results can be expected in European comparison.

Presentation 3: Instrumentalization of Emotion During the 2016 U.S. Presidential Election – A Neopragmatist Analysis of the Presidential Nominees’ Media Communication

Presentation 3 author: Sheena F. Bartscherer Presentation 3 abstract:

Despite Hillary R. Clinton being the clear favourite to win the 2016 U.S. election, Donald

J. Trump won the race for the Republican party. Trump’s campaign was defined by “nationalistic” and “populist” narratives, following a strategy of political marketing designed to emotionalize the political discourse in a way usually reserved for autocratic regimes (cf. Gillies 2018: 2). Following neuroscientific theories and practices, we

scrutinize this postulated emotionalization and the implied conflation of ‘populism’ with ‘emotional’ (political) communication (Arroyas Langa 2019), by focusing on the

occurrence of three key emotional stimuli (sex, fear & violence) within Clinton’s and Trump’s public speeches, interviews, televised debates and social media (twitter & Facebook). Alongside neurolinguistic dictionaries representing these three emotional stimuli, we establish a neopragmatist discourse analysis, identifying each campaign’s distinctive argumentative pattern and the role emotional stimuli play within them. The applied analysis method extends the Pragmatic Sociology of Critique (PSC) by measures of suggestive actions (past and future references thereof). Within our empirical data, we detect no significant quantitative differences between the two campaigns’ instrumentalization of emotions: both are directly and indirectly employing them within their main arguments. Instead, we discover multiple significant qualitative differences:

Trump’s ‘populist’ deemed campaign mainly employs violent stimuli, exhibits a simple

and repetitive argumentative structure and an overall negative sentiment. Rather than using emotions as identifiers for ‘populist’ discourse, our findings suggest focusing on actors’ argumentative patterns and strategies, their argument’s sentiment (positive/negative) and the specific type of emotional stimuli employed (sex, fear or violence).

Presentation 4: Between Baby and Blood: Interpreting Accounts of Pregnancy Loss on You-Tube

Presentation 4 author: Julia Böcker M.A. Presentation 4 abstract:

Early pregnancy loss often leads to ‘disenfranchised grief‘ (Doka 2002). Whereas women might experience labour pain and the loss of their baby, and grieve according to that, a common response in so-called western cultures is that ‘it wasn’t a real baby yet’ (cf.

Layne 2000). In my paper, first, I discuss the relation of materiality, meaning and their relevance for (re)shaping feelings in the context of pregnancy loss experiences by drawing on findings from my dissertation project, conducted in a Grounded Theory manner. Second, I will discuss sequential video analysis in interpretation groups as a methodical approach that proved suitable, on the one hand, to clarify the nexus between materiality and meaning. Aiming to understand the emotional culture surrounding pregnancy loss in Germany, and assuming that it shapes the initial ‘framing of the loss’ (Jakoby 2015), I sought to analyse the intimate process of miscarrying –

respectively ‘losing something’ – itself. For that You-Tube videos by women who document and share their experience of a recent or even on-going miscarriage turned out to be rich first-hand data. On the other hand, the approach revealed cultural norms and ‘feeling rules’ (Hochschild) concerning pregnancy loss by including emotional responses co-interpreters had while watching the videos and interpreting the data (e.g. being disgusted). It also turned out, opposing to these feeling rules it is a motivation for many video-makers to share their story and thus reframe their individual loss experience.

Doka, K. J. (Ed.). 2002. Disenfranchised grief. New Directions, Challenges, and Strategies for Practice. Champaign: Research Press; Jakoby, N. 2015. The Self and Significant Others. Toward a Sociology of Loss, Illness, Crisis & Loss 23(2): 110–128; Layne, L. L.

2000. “He Was a Real Baby with Baby Things”. A Material Culture Analysis of

Personhood, Parenthood and Pregnancy Loss, Journal of Material Culture 5(3): 321–345.

Presentation 5: Core Relations Themes as a method for reconstructing emotion from text

Presentation 5 author: Christian von Scheve Presentation 5 abstract:

Text can be emotionally captivating, even if it does not contain a single emotion word, such as anger or joy. Some of the emotion-inducing qualities of text are found at the structural level, in phonetics, or the use of figurative language. In sociology, however, we are often confronted with ad-hoc narratives produced by interviewees or speakers in public discourse that lack a finely crafted aesthetic dimension. Reconstructing

articulations of emotion or inferring a speaker’s emotional experience from text as well as assessing a text’s capacity to induce emotions in recipients are methodologically challenging. I suggest that appraisal theories of emotion in conjunction with interpretative methods provide valuable insights for this task. In a nutshell, appraisal theories assume that emotions result from interpretations of events regarding a

person’s beliefs and desires. Lazarus developed the concept of “core relational themes”

(CRT) which reflect the basic meaning structure of a range of discrete emotions. I

propose that to reconstruct emotions from text or to assess the emotional impact of a text, identifying CRT in a text is essential. Qualitative inquiry generally aims at reconstructing meaning from text. Interpretation presupposes that at least some common ground and intersubjective understanding be shared between interpreter and the interpreted. Given that a text contains, for example, autobiographical or other self-

related narrative that reveals a speaker’s beliefs and desires, and that the researcher can reasonably assume to share stocks of knowledge with the speaker, one can identify CRTs and related emotions. Likewise, it should be possible to estimate likely emotional reactions of a given audience to which we can attribute certain beliefs and desires.

Although the proposed approach does not provide insights into actually experienced or

perceived emotions, it allows establishing emotional “frames” or “repertoires” along

which individuals and audiences alike make sense of the world.

15:30-

16:45

Session 9: Innovations and challenges in qualitative research: A real-life approach

Session chair: Inga Gaisauskaite, Claire Wagner

Presentation 1: Quality Criteria for Online Qualitative Research Presentation 1 authors: Marlene Schuster, Melanie Hense Presentation 1 abstract:

The global corona pandemic is currently prompting many researchers to transform their research projects into online concepts. For qualitative social research, finding ways of collecting and analyzing data that do not compromise the quality of the results, is especially challenging due to the limited possibilities (e.g. missing personal contact). This research therefore explores the extent to which quality criteria of qualitative social research are already being discussed in the context of online research. To address this topic we conducted a systematic literature review of academic journal articles dealing with the quality standards of qualitative social research and qualitative online research in the period 2000 up to now. First results show that quality criteria applied to qualitative research in the online context is only marginally addressed, which may mean that (1) there is no special need to develop new specific quality criteria for the online context, (2) as with the “general quality criteria”, there actually isn’t and won’t be a consensus within the scientific community, or (3) that this is (still) a blind spot in quality assurance. Thus, this literature review aims at providing an overview of the status quo of quality criteria for online qualitative research. Consequently, in times of pandemia this should be an impulse for researchers to further consider the quality criteria of qualitative social research with specific inclusion of the online context.

Presentation 2: Integrating Q Methodology within In-depth Interviews and Focus Groups: Making Sense of Europeanization Discourses in Georgia

Presentation 2 author: Lia Tsuladze Presentation 2 abstract:

The presentation focuses on the first occasion of integrating Q Methodology within in- depth interviews and focus groups in the study of political and popular discourses on Europeanization in Georgia. Despite its quantitative focus, Q Methodology is used with a rather small sample grasping subjective views of target group members and enabling a deeper analysis of individual cases. Its correlation and factor analyses can be a valuable addition to the qualitative data analysis enabling its validation and fostering its further understanding.

Driven by the above considerations, Q Methodology has been integrated in the course of

in-depth interviews with political and intellectual elites and focus group discussions with the population residing in various regions of Georgia. The developed Q grid consisted of 30 statements derived from our former research on the political and popular discourses on Europeanization in Georgia. All key discourses offered by politicians and population were gathered from the former research resulting in more than 100 statements, out of which the most essential 30 statements were selected to be used in the current research. The research participants first ranked the statements in Q grids and then commented on the most important statements they had placed on the extreme positive and negative poles. The limited number of cells pushed the participants to select the priority statements explaining the rationale behind their choices, accompanied by the follow up open-ended questions by the researchers. The two components of the gathered data were subject to Q analysis and qualitative content- and discourse analyses, respectively.

The proposed presentation discusses the construction and application of the described research instrument, and illustrates how the integration of Q methodology within in- depth interviews and focus groups has resulted in more comprehensive data and

enabled a better understanding of the participants’ multifaceted perspectives on Georgia’s Europeanization. The very same approach will be applied in the ongoing Horizon 2020 RIA project entitled “MEDIATIZED EU: Mediatized Discourses on

Europeanization and their Representations in Public Perceptions” in which the author

leads the Research Work Package.

Presentation 3: Challenges in conducting longitudinal qualitative research among parentally bereaved African children

Presentation 3 author: Mienke van der Brug Presentation 3 abstract:

Parental loss is recognized as one of the most stressful events in the lives of children and adolescents. In Namibia, a fifth of all children have lost one or both their parents, partly as a result of the HIV pandemic. There is a need to understand children’s bereavement experiences using qualitative methods, especially from a longitudinal perspective.

Longitudinal designs capture the dynamic and evolving processes of grief in various life stages, and give insight in the intersection between structural factors and individual lives over time. In this longitudinal, ethnographic study on parental loss, a group of 14 orphans took part as children (aged 9-12 years, in 2003/2004) and as adolescents (18-21 in 2010-2013). I propose to study this group again as adults (28-32 years in 2023). In this presentation, I will discuss the (methodological) challenges encountered during the first two phases of this study. For example, the different approaches towards interviewing participants in different life stages, and difficulties in tracing participants including its ethical issues. I will also reflect on possible challenges and opportunities which can be expected when studying these participants again as adults, and reflect on how longitudinal research affects the quality of the data.

Presentation 4: Interviewing adolescents through time: balancing continuity and flexibility in a qualitative longitudinal study

Presentation 4 author: Susanne Vogl Presentation 4 abstract:

Qualitative longitudinal research (QLR) has great potential for elucidating processes and change over time. Despite the growing interest in QLR, methodological and practical challenges require further reflection. In this contribution, we reflect on two major issues in interviewing adolescents in QLR: panel maintenance and changes in the research set- up, including interviewing technique, content and interviewer (dis)continuity. Based on

experiences from a panel study on understanding how young people’s opportunities in life are shaped during a transitional stage (‘Pathways to the Future’), we present methodological and pragmatic decisions, rationales and lessons learnt to inform future qualitative longitudinal studies. We show how change is omnipresent in QLR practice, and how it demands researchers’ openness and flexibility as well as finding a balance between continuity and adaption. The process can be challenging, but it also offers

opportunities.

17:00-

18:15

Board Meeting

09:00-

10:15

Session 10: Non-binary sex and gender in general population surveys: Current developments

Session chair: Verena Ortmans

Presentation 1: Beyond the gender item – constructing a gender inclusive questionnaire on sexual health

Presentation 1 authors: Kathrin Gärtner, Irina Igerc Presentation 1 abstract:

For a study on sexual satisfaction, chronic illness and sexual dysfunction we wanted to construct a questionnaire which was not only non-heteronormative but also non-mono- normative (not presuming monogamous relationships) and especially gender-inclusive. As we soon realized gender inclusiveness does not only mean to include one or two questions on gender (and sex assigned at birth) but had implications for other questions as well. For example, in the question regarding sexual attraction we could not just ask if one was attracted to men and/or women. We also had to put a special effort into item formulation as an all gender-inclusive language had to be used (which is in German not as easy as in English). Finally, we could not rely on specific scales on sexual health or sexual beliefs with versions for women and men (which turned out be hetero-normative and mono-normative in many cases). Regarding the gender questions, we consulted a group of trans-men/people with a non-binary gender identity who favored an open question regarding gender. However, as we were afraid to confuse people who have not heard much about gender identity yet we decided to use a two question- format, which includes different options as for example trans-male, trans-female or non-binary but also an open option which was used by two participants of our study. In our presentation, we are going to discuss our decisions regarding items but will also show some results and share experiences with our questionnaire

Presentation 2: Gender identity and sex: contexts of data collection Presentation 2 authors: Eleanor Scott-Allen, Andrew Nash Presentation 2 abstract:

Sex and gender have been traditionally seen as similar concepts for the purposes of data collection, but this has been increasingly changing with an emerging need to understand the transgender community. Both sex and gender data provide valuable insights into both the general population and specific populations, on topics such as health and equality monitoring.The Office for National Statistics (ONS) has been conducting a programme of work to examine how we can collect data that answers the needs of our stakeholders whilst providing value to the public.As part of this work, the ONS has conducted research into the various national contexts of gender identity data collection as well as how sex and gender data is collected throughout the world.The presentation considers a number of issues that impact how sex and gender data is collected within a national context including existing administrative or survey infrastructure, differing legal and political contexts and the impact of language.

Presentation 3: Perception and acceptance of a third gender category in Germany _ Results of a web probing experiment

Presentation 3 authors: Verena Ortmanns, Patricia Hadler, Cornelia Neuert Presentation 3 abstract:

Gender is a central socio-demographic characteristic and nearly every survey asks respondents on it. Until now, this characteristic typically is measured by a simple question (e.g., “What is your gender”?) with two answer categories (“male” and

“female”). By a change in German law, an additional designation for intersex people has

been introduced in the official register of births. Besides leaving that section blank on birth certificates, intersex people now have the option to select a third category

“divers”. This has led to discussions on how to deal with this new category in surveys. For one, it is important that respondents have a common understanding of the category.

Secondly, respondents might be confused by or perceive the new category as a

provocation, and might break off the survey or change their response behaviour. In this paper, we ask: How is the inclusion of a third answer category for gender perceived in a general population survey? Moreover we ask whether question wording influences which construct (biological sex or gender) respondents think of. We conducted an online experiment and applied web probing. We asked several comprehension probes to investigate whether respondents are aware of the rationale behind the third category and how they understand this category. Additionally, we tested different wordings of the sex/ gender question. First results show that the large majority of respondents are familiar with this term and that nearly two-thirds of the respondents have a clear or approximately correct understanding of it. However, the results also indicate that most respondents cannot properly distinguish between the constructs of biological sex and gender identity because it appears to be the same for them. We discuss the results and the limitations of this study as well as the implications for asking the gender question in Germany and in intern.

Presentation 4: Gender role attitudes in the Philippines and elsewhere Presentation 4 authors: Harry BG Ganzeboom, Linda Luz Guerrero, Iremae Labucay Gerardo Sandoval

Presentation 4 abstract:

The Gender Role Attitude (GRA) index is a 5-11 indicator instrument to measure differences in opinions about the proper division of household tasks. The instrument has been used in ISSP’s Gender&Family modules since 1988, as well as other surveys, such as EVS, ESS and US-GSS. Using an multiple-correspondence analysis, the ISSP1994 data on GRA were critically examined by Blasius (2006), who scorns particularly the Philippine data for being completely inconsistent, and concludes that “it makes no sense to

compare the data (…) from the Philippines with that of any of the other countries”. We take up the challenge by examining the GRA data from a broader and less technical design. First, we analyse measurement quality in all four ISSP Gender&Family modules, which brings a considerably broader database of countries and time-points. Second, we examine measurement validity not internal associations of the instrument, but in relationship to external validation criteria, such as gender and cohort. Third, we also seek to separate validity from reliability and test whether an instrument with poor (but some) reliability can still bring out structural relationships with the validation criteria.

Our first finding is that the Philippine GRA-data indeed have low reliability, but are far from randomly generated. This finding is consistent across waves, and is replicated for a number of countries that are similar either in socio-economic development or geographical/cultural location: Mexico, Thailand, India, Japan and Taiwan. The problem may be more substantive than technical. Second, we find that despite low reliability, the cross-national ranking of the Philippines as very gender-role conservative is consistent between ISSP waves, which confirms that data with low reliability can still contain

substantive meaning. Third, we find that the usual determinants of differences in GRA

(e.g. women and younger cohorts being less conservative) work in the Philippines just the same as elsewhere.

10:30-

11:45

Session 11: (a) Trust in science / (b) Framing and Taboo in research

Session chair: Inga Gaisauskaite

Presentation 1: The great unmentionable”: the rhetorical construction of taboo on

death and is origins.

Presentation 1 author: Irina Romanova Presentation 1 abstract:

The idea that death in today’s society is a taboo subject, not to be mentioned in polite conversations or in front of the children, has become commonplace. One can often find it in the press, in popular literature on death, dying, and bereavement, and on the

websites of many palliative care organizations. The concept of a “death café”, an informal gathering of people to speak about death over a cup of tea that originated in 2004 in Switzerland and spread to 50 countries around the world, is also based on the idea that death is a sensitive topic that requires a specially set time and place for discussion. In the words of its founder, Bernard Crettaz, death café is aimed at breaking the “tyrannical secrecy” surrounding the topic of death.

The origins of this idea can be traced to the scholarly literature on death and dying of the 1950s, more specifically the early publications of Herman Feifel, the founder of the modern thanatology. “Breaking the taboo on death” is considered to be his main contribution to the field as it was stressed in the citation of his 1988 Award from the American Psychological Association. This paper discusses the rhetorical strategies used in talking about the repressed subject taking as an example the 1959 volume “The Meaning of Death” edited by Herman Feifel. In my analysis I use the methods of Rhetorical Social Psychology developed by Billig (1987; 1991). I also draw from Psychoanalytic Discursive Psychology (Billig, 2006) when comparing how two subjects, death and sexuality, were discussed by Herman Feifel and Sigmund Freud respectively.

Presentation 2: Environmental compliance management of production companies – Disclosure identified through methods of observation and scraping of webpages Presentation 2 authors: Karsten Boye Rasmussen , Heiko Henning Thimm Presentation 2 abstract:

Commercial companies producing goods are also producers of environmental effects. Typically, state and supra-state organizations like the European Union and United Nations are goal setting. Corporate environmental compliance management (CECM) is now a necessity for large companies. Environmental, social, and governmental responsibility and the according ESG measurement implies that companies seek to demonstrate environmental responsibility and sustainable behavior. The public are becoming continuing more alert to the environmental agenda, the state goals, and company performance. Consumers are demanding environmental behavior from their suppliers and have focus and perform activities in initiatives of sustainability. While earlier on the trammels of strict environmental compliance could be seen as a hindrance of company free enterprise, it can now be viewed as a positive force as early adopter companies gain a strategic competitive advantage because other companies will soon have to comply to the new norms. Companies can also have an advantage in publicizing themselves as responsible actors with a focus on sustainability and efforts towards

environmental compliance management. In order to investigate this tendency of

openness and disclosure this paper employs a method of website observation of a random selection of the largest production companies. Website observation is used because the website is the company main information channel for both customers, suppliers, and the general public. Observation of websites is our preferred method for research both as a relatively inexpensive way to obtain current data but primarily as an unobtrusive method that evade the perils of questionnaire and interview techniques. In particular the online observation avoids the otherwise common high levels of nonresponse. The employed method of human observation contains a systematic data collection that is secondly attempted validated through scraping of company webpages. The research summarizes a richness of variables into formative indices for further analysis and categorization of production companies.

Presentation 3: Is science at risk? The bad influence of politicization and industrialization for scientific rigor

Presentation 3 author: Ana Muñoz van den Eynde Presentation 3 abstract:

Science is at a crossroads, conditioned by social factors that arise and develop at high speed at a critical moment in history and are generating some dynamics that involve the establishment of new practices intrinsically strange to the culture that has traditionally ruled science. Among the pairs of forces contributing to distort scientific activity, in this contribution we highlight two. One confronts politicization and discourse: in the face of “scientific dogmatism,” for which nothing exists if science has not demonstrated it, emerges a worrying “uncritical scepticism” which is fostering the rise of denialist movements. The other dangerous dynamic associate industrialization and content, as it is contributing to science being subordinated to political and corporate interests that do not give enough value to the production of knowledge in itself. On the other hand, the vision of science as a “blind” evolutionary process, without prior knowledge of its aptitude and usefulness, is counteracted by a force of the opposite direction oriented to “design” it in order to serve interests that do not imply necessarily the obtainment of valid and reliable knowledge. Both dynamics contribute to defining the context in which is framed the analysis of a scientific article that has proved to be false. In it can be identified the dangers stalking science, especially the lack of quality and objectivity that are, in turn, the result of the ambition to publish a supposedly ground-breaking article easy to “sell” in a determined climate of opinion. From this analysis generates the need to reflect on the following issues: why have failed the supposedly infallible filters of the peer review system? What are the reasons of the magazine not to retracted it? Can we trust the articles that are published? If science loses its objectivity, what do we have left?

Presentation 4: Research partnerships between Indigenous and non-Indigenous researchers

Presentation 4 authors: Rosalind Edwards

Presentation 4 abstract:

International research initiatives that seek to address global challenges often require collaborations between western-based non-Indigenous researchers and Indigenous researchers living in and working with their communities. Despite best intentions, these collaborations may reproduce colonial approaches to knowledge production and use.

They may position Indigenous researchers as junior members of the team, and research findings can construct Indigenous peoples as deficient in comparison to non-Indigenous

norms, rather than also tracking the reproduction and accumulation of privilege and identifying resilience.

All research questions and methodologies are grounded in the specificities of people’s world views, and research as an activity occurs in a set of historical, political and social contexts. But some world views, methodologies and methods are accorded more legitimacy and privileged over others. Developing equitable Indigenous and non- Indigenous research partnerships means challenging perceptions of the relationship between Indigenous and Western knowledges, and what counts as knowledge and appropriate research practice.

In the ‘Indigenous and Non-Indigenous Research Partnerships’ project, Indigenous researchers: Helen Moewaka Barnes (Massey, New Zealand) and Deborah McGregor (York, Canada) and non-Indigenous researchers: Rosalind Edwards (Southampton, UK) and Tula Brannelly (Bournemouth, UK) have worked with communications expert Christine Garrington (Maltstore Communications) and graphic artist (Olivia Hicks, Glasgow) to develop a set of resources to help non-Indigenous researchers think about how they can approach working with Indigenous researchers and communities. These include an illustrated audio panel discussion of good and bad practice, blogs from Indigenous and non-Indigenous researchers about issues they have faced, and a comic

about effective collaboration.

12:00-

13:30

Session 12: Natural Language Processing: a New Tool in the Methodological Tool-Box of Sociology

Session chair: Renata Nemeth, Ildikó Barna

Presentation 1: An analysis on the framing of Trianon and the Holocaust in Hungary using Natural Language Processing

Presentation 1 authors: Árpád Knap, Ildikó Barna Presentation 1 abstract:

In 2020, Hungary was commemorating the round anniversary of two important historical events of the 20th century. The first is the centenary of the Treaty of Trianon, and the second is the 75th anniversary of the liberation of the death camps and the end of the Holocaust. Both events have had a great impact not only on Hungarian but on world history. However, what is more important for our research is that the evaluation and the memorialization of Trianon and the Holocaust are still among the most divisive topics in Hungary, dividing the different sides of the political spectrum. This division is present in both political and public discourses. The increasingly polarized public sphere in Hungary provides a rich empirical base for sociological research and makes possible not only to describe the framing and the interpretation of the above topics but also to analyse the similarities and differences of memorialization of distinctive political sides.Since the transition of 1989, an intensive discussion has been present about both events in the political and public sphere. Not only the anniversaries mentioned about but also the intense and controversial memory politics of the present government resulted in abundance of online content. The analysis of the vast amounts of unstructured online textual data demands new tools; one of them is Natural Language Processing (NLP). In our research, we apply NLP methods on a massive corpus of recent Hungarian online content from different sides of the political spectrum. Using NLP’s topic modelling, we examine the structure, the main topics, and discourses about Trianon and the Holocaust, and furthermore, we identify the key differences in memorialization and the framing on different sides of the political sphere using word embedding models.

Presentation 2: Understanding online discursive framing of depression via topic models

Presentation 2 authors: Renáta Németh, Domonkos Sik, Eszter Katona

Presentation 2 abstract:

Social sciences have criticized the reductionist biomedical discourse, which has been dominating expert discourses of depression for a long time (Kangas 2001, Kotliar 2015, Sik, 2018). As these discourses determine the horizon of attributions and interventions, their lay interpretation plays a central role in the coping with depression.

To map the patterns of online representation of depression, we analyzed the most popular English-speaking forums (n~70,000). Although there has been previous research which examined the framing of depression in online health forums using text mining methods (e.g. Pan et al., 2018, Feldhege et al., 2020), to our knowledge our research is the first which provides a comprehensive analysis by expanding the horizon of both the amount of collected data and the applied analytical tools.

In a previous paper we used human annotation and supervised learning to classify posts according to their framing type (Németh, Sik, Máté, 2020). In the present paper, we discuss the potential for Latent Dirichlet Allocation (LDA) topic models in this field (Németh, Sik, Katona, 2021). LDA was complemented with a qualitative approach.

Following the evaluation of various models 18 topics were differentiated. They reveal the monolog (attributions and self-disclosures) and interactive (consultations and quasi- therapeutic interactions) patterns. The topics were evaluated from the perspective of the biopsychosocial model.

Presentation 3: Uncovering the discursive construction of students in higher education

Presentation 3 authors: Rakovics Márton, Réka Tamássy Presentation 3 abstract:

The purpose of our presentation is to introduce our research on the discursive construction of students in higher education on university websites. In addition to our findings, we discuss the advantages and difficulties of using mixed methods in such an analysis. Applying a discourse analytic approach, we theorize that the representation of students on university webpages has a sustaining, legitimating and constructing effect on the identity of university students in a social context. Mixed methods as a methodology makes it possible to manage a large corpus but requires a precisely defined research object due to the extensive overlap in used expressions in the specific language of the analysed field. Although this approach is unusual for discourse analysis since it depends on presuppositions about the text, it allows a deeper analysis of themes that stand out from the more comprehensive description of student representation extracted with text mining. Our corpus-based discourse analysis was carried out on the top 100

universities of the Times Higher Education’s 2019 list of business & economics world university ranking. Three sections of the websites – ‘about us’, ‘research’ and ‘study’ – were gathered and analysed. Student portrayal is inseparable from both the representation of the university as an institution and the linguistic image of the purpose and process of learning. Thus, the research concentrates on the textual realization of the aforementioned phenomena by focusing on the linguistic structures and used

expressions concerning the referencing techniques, the future of students, students’ roles, student representation as pupils or consumers, and the portrayal of learning. To validate both our findings and our methodology, we compared student profiles

corroborated by discourse analysis with the uncovered polysemy of ‘student’ using a

distributed vector representation of words.

Presentation 4: Detecting Latent Changepoints with Topic Model Presentation 4 authors: Koltai Júlia Anna, Anna Molnár Presentation 4 abstract:

The sudden appearance of the coronavirus shook the world in every way. Measures related to the virus have a strong impact on people’s daily lives, so information about the relatively unknown virus is of particular importance. All new knowledge forces people to rethink previous actions and behaviors, so – especially because of the limitations of physical encounters – these discourses also actively appear in social media. In our research, we examine what were the main vaccination-related narratives in the virtual space of social media, including the main points where there was a change in the narratives based on the texts – and what real-time events can be linked to these changes

. The analysis is performed with natural language processing, including the basic and extended versions of LDA topic modeling, as well as HBTM topic modeling.

Presentation 5: Gender disparity in an online video game live-streaming community

Presentation 5 author: Tóbiás Dániel Presentation 5 abstract:

As we are on the way to the ubiquitous networked society, more and more social activities move to the online surface. Despite many of its advantages, gender inequality persists in online media. Online gaming is not only recreation but a popular spectator sport as well, and it is the reason, why online video game live-streaming is rising and forging global communities. In Twitch.tv, what is the most popular live-streaming media platform among gamers, objectification and gender disparity is still a problem. Using Twitch’s API services, this study focuses on 2 game’s communities and using NLP methods, to reveal the suspected objectification or any gender disparities towards streamers in chats.

13:45-

15:00

Session 13: Teaching qualitative, quantitative and mixed methods in social science programs

Session chair: Inga Gaisauskaite, Claire Wagner, Florian Berens

Presentation 1: Teaching qualitative, quantitative and mixed methods in social science programs

Presentation 1 author: Florian Berens Presentation 1 abstract:

In most cases, study programs in the social sciences include a basic education in qualitative and/or quantitative methodology. In these courses, prospective researchers of the social sciences learn the basics for their future work. In addition, these courses provide basic knowledge that is also in demand in society outside of science and an important part of (vocational) training of the students. It is therefore important to scientifically examine methodological training in the social sciences and to reflect it among social scientists. In this context, this session aims to address the issue of university teaching for the training of qualitative, quantitative and mixed methods. The session includes both the theoretical debate on what to teach and the discussion of empirically anchored knowledge about how to teach. Those interested are invited to enrich the session with…

1.  … theoretical contributions on what the aims and contents of the methodological training of undergraduate students should be. Welcome in this strand are contributions that reflect upon the traditional methodological training as well as contributions that focus on new challenges for methodological training, e.g. technical innovations, new data forms, alternative facts.

2.  … empirical studies that show ways to improve methodological training at universities.

In this strand there is the possibility to report on the application of general pedagogical

or psychological findings in methodological training in order to demonstrate and reflect their subject-specific implementation. Especially welcome are own empirical studies on teaching in methodological training.

Presentation 2: Applying research methods using multidisciplinary project-based learning

Presentation 2 authors: Claire Wagner, Jacques du Toit Presentation 2 abstract:

Two departments at the University of Pretoria, Psychology and Town and Regional Planning, collaborated with an external research partner to conduct research on an environmental topic. Master’s research psychology and fourth year urban planning students worked with the science council’s project team and their respective course

lecturers in groups on different phases of the project. This provided a space for learning about research in the real-world as part of a multidisciplinary team and a chance to engage with an external research partner. The aim of this paper is to (1) describe how the multidisciplinary project-based learning approach was used to assist the two groups of students to apply research methods they had been taught and (2) provide insights

into the students’ experiences of the approach. Qualitative data was collected about the students’ learning experiences. Although both groups experienced operational challenges they expressed their appreciation for the rich learning opportunities presented to them in dealing with these challenges. Students articulated how the project allowed them to see the practical application of what they have learnt about research methodology and that they gained invaluable experience on issues to consider in research instrument design, sampling, recruitment and data management. They also developed an appreciation for the importance of project management. The students reported feeling more equipped to conduct research in industry as a result of having participated in the project. This study suggests that multidisciplinary project-based learning has significant potential, but that future programmes could seek out opportunities where students are able to participate more fully and independently in the research project. The richest learning opportunities were found in situations where students were required to actively problem-solve and address issues encountered in the field.

Presentation 3: Mapping the terrain: Using instructional technology for teaching research methods

Presentation 3 authors: Claire Wagner, Barbara Kawulich, Mark Garner Presentation 3 abstract:

Ease of access to the Internet and the pressure to accommodate increasing numbers of students who are also diverse in geographical and demographic spread has resulted in higher education extending its educational reach into the virtual world. The instruction of social research methods has followed suit. It could be argued that using technology as an alternative approach to traditional teaching formats may also appeal to students who tend to hold negative attitudes towards and experience anxiety about research methods courses. This paper will map the terrain of using instructional technology for teaching social research methods. Journal articles that describe this topic – ranging from converting face-to-face courses to online formats to the use of virtual worlds such as Second Life – are examined. The benefits and challenges of using instructional technology for teaching research methods will be presented and recommendations made for those who plan to take up this educational approach.

Presentation 4: Flipping service learning by implementing it in research seminars on empirical methods

Presentation 4 author: Florian Berens Presentation 4 abstract:

Traditionally, service learning is an approach from the field of experiental learning, in which students gain experience in practical contexts. In Service Learning, the constructive output that arises for actors outside the learning group is decisive for style. Classical Service Learning of this form is based on a social need and uses students as an answer to this need. In contrast, student-oriented education should start from the learning goals for the individual student and find answers to the question of which methods are suitable for achieving these goals. The implementation of Service Learning should therefore not be the starting point of university curriculum planning, but at best its result. Service Learning understood in this way is not a learning goal, but a supplement to professional and interdisciplinary teaching goals of university education. At the University of Göttingen, the presenter has implemented service learning of this understanding in a seminar on research methods in quantitative social science. The Tafel Göttingen, an institution that distributes leftover food to the needy, was won as a cooperation partner. In the seminar, the partner’s clientele, its donors and its image

could be examined in order to gain valuable information for the partner’s further

development of their work. The presentation will use this case to illustrate the benefits of cooperation for all partners involved and in particular discuss questions of freedom of research in this context.

15:15-

17:00

Session 14: General Population Surveys on the Web: Designs and Usability of Data. Insights from the European Values Study

Session chair: Michèle Ernst Stähli, Tobias Gummer, Gudbjorg Andrea Jonsdottir, Vera Lomazzi, Ruud Luijkx, Michael Ochsner, Alexandre Pollien

Presentation 1: Identifying and addressing representation issues of different survey designs: Evidence from an experiment with the European Values Study 2017 Presentation 1 authors: Jessica Herzing, Michael Ochsner, Alexandre Pollien Presentation 1 abstract:

The paper presents a framework to analyse representation issues of different survey designs and applies it to compare five different designs of the same survey, the European Values Study. Designers of surveys that are repeated over a long period of time, being it cross-sectional or longitudinal, are confronted with changes in the habits of their population and consider a change of the design or mode to adapt to such societal developments. One important question is whether the different surveys are similarly successful in reaching the population or whether design, and (d) a short push- to-web survey using matrix design with a follow-up survey and compare it to face-to- face version of the same survey. Using register data (internal benchmark) and official statistics (population benchmark) and other general population surveys (external benchmark), we identify differences in the samples and their potential consequences for different uses of the data.systematic biases in respondents result from changing the design. Our framework rests on the assumption that every sample is biased and that the importance of the bias is dependent on the context. We investigate issues of representation of (a) a long push-to web survey announced as long, (b) a long push-to web survey announced as short, (c) a short push-to-web survey using matrix design, and

(d) a short push-to-web survey using matrixdesign with a follow-up survey and compare

it to face-to-face version of the same survey. Using register data (internal benchmark) and official statistics (population benchmark) and other general population surveys (external benchmark), we identify differences in the samples and their potential consequences for different uses of the data.. We then conclude by suggesting measures to account for such representation issues and general recommendations.

Presentation 2: How and when to use multiple imputation when analysing data from a cross-cultural survey in matrix design or with high amounts of missing data Presentation 2 authors: Michael Ochsner, Jessica Herzing, Tobias Gummer

Presentation 2 abstract:

Different societal developments render web surveys more attractive and feasible than interviewer-based surveys, like telephone or face-to-face surveys. However, not much is known how to field general population surveys on the web because the length of such general population surveys usually doubles the suggested maximum length of web surveys. One solution for this problem is to reduce survey length by adopting a matrix design, where the questionnaire is split into modules and not every respondent answers every module. This design, however, comes with the disadvantage that there are missing data by design and statistical power is reduced. When using many variables from different modules in an analysis, it can even happen that no observations are left for the model. In this presentation, we show how to use data from a matrix design in cross- cultural research for different situations. Comparing data from different designs (face- to-face, long web survey and web survey in matrix design) and different examples of research questions, we show when and how to apply multiple imputation when using cross-cultural data collected in matrix design.

Presentation 3: Measurement equivalence across different designs of the European Values Study 2017

Presentation 3 authors: Vera Lomazzi, Daniel Seddig, Presentation 3 abstract:

The implementation of mixed mode survey designs is a relevant innovation that can support the development of survey research in the next future: it’s cost-effective, potentially decreases coverage errors and non-response errors. But which are the practical consequences for data users and comparability?

The use of mixed mode can introduce a method bias threating the measurement equivalence across modes and cultural context. For example, the presence or absence of an interviewer, the different ways the question is provided to the respondent, the layout on the screen, or some slight changes in the wording adopted across modes of data collection can affect the question-answer process and risk to turn out in measurement constructs that are not equivalent between modes. When the studies involve cross- cultural comparative aims, these mechanisms are even more amplified by cultural differences in item interpretation, translation issues, etc.

Using not equivalent measures can bring scholars to misleading substantive results because the assumption of comparability is violated. Investigate whether data collected by different mode can be used all together and comparatively is therefore crucial.

By focussing on constructs such as gender role attitudes that – according to the extant literature- are particularly sensitive to measurements biases, the contribution adopts state-of-the-art techniques, including the alignment procedure, to assess equivalence between modes in each country and across the whole set of countries included in EVS2017.

Presentation 4: Potential consequences of splitting a questionnaire: order effects,

battery splitting and the respondents’ mood

Presentation 4 authors: Alexandre Pollien, Oliver Lipps, Jimena Sobrina Piazza Presentation 4 abstract:

Changing the mode of a survey or splitting a questionnaire has consequences on how questions are answered. This presentation focusses on two such issues: First, the change from face-to-face to a web survey comes with a change of perceived length as web surveys are considerably shorter and respondents do not expect to answer a one-hour questionnaire on the web. Second, splitting a questionnaire affects the order of the questions as not all questions are asked. The meaning of a question, however, is not entirely embedded in the question itself. Many factors may influence the way respondents understand and answer a given question. The meaning of questions, as well as the ideas and the standards of comparison respondents consider to answer them, are all influenced by questions previously answered in the questionnaire. Changing from a one-hour face-to-face to a 25-minutes web survey in matrix design thus might raise problems of comparison when a same question is asked in different splits of the questionnaire.

In this presentation, we investigate the influence of such effects of a transformation from a one-hour face-to-face to a web survey using the European Values Study 2017 as an example. We focus on the effects of question order, battery splitting and the mood (emotional state) of respondents across different designs of the EVS experiment, including two long versions of the same questionnaire with variations in question ordering and a matrix design. We first undergo a global assessment of the effect of questionnaire design on the mood of the respondent, his/her interest in the actual survey and his/her propensity to respond to a next survey (follow-up). We then analyse some specific issues of splitting questionnaires in more detail and provide guidelines how to consider such issues when analysing data from matrix designs.

Presentation 5: Effects of mixing modes in general population surveys: Insights from the European Values Study 2017

Presentation 5 authors: Tobias Gummer, Pablo Christmann, Christof Wolf Presentation 5 abstract:

Conducting design experiments as the EVS did has implications for data distribution and use. Providing researchers with data sets that were collected in different survey modes raises the question on whether mode effects exist. In other words, when drawing conclusions from these data it is important to know whether respondents differ conditional on their mode of participation. We present a case study based on the EVS experiments and compare 138 substantive items of the EVS between (1) interviewer- administered face-to-face and (2) self-administered mode. In our experiment, the gross sample was randomly divided into subsamples that were either invited to participate in the EVS via face-to-face or self-administered interviews. To not only gauge whether differences exist but also their magnitude, we employ a diverse set of indicators of inequality between both samples.

Presentation 6: Measurement invariance in comparative research: an example based on the European Values Survey data

Presentation 6 authors: Clelia Cascella

Presentation 6 abstract: This paper aims to discuss measurement invariance in comparative research, at different levels of locality (i.e., national and sub-national levels). The measurement of gender attitudes towards gender and gendered roles both in family and in society has been used as an example. Data collected by the European

Values Survey (EVS) – that are statistically representative of the population of each European country at both national and sub-national levels (European Values Survey, 2020) – have been used to measure the variability of gender attitudes between European countries and regions. The paper thus reports systematic variations of gender

attitudes making regional and national comparisons simultaneously visible on the same scale across Europe. In doing so, the measure of attitudes towards gender inequality has been broadened by adding EVS items not employed in previous research to measure gender attitudes, in the attempt to better capture modern views of cultural inequality. To validate the scale, the Rasch model was employed (Rasch, 1960/1980). The differences between the Rasch model and other analytical approaches frequently employed to develop similar scales (such as the factorial analysis) have been discussed and used to understand the contribute of the Rasch analysis to the establishment of measurement invariance (Engelhard, 2013).

Measurement invariance has been discussed also in a longitudinal perspective. To this end, EVS data collected in 2008 and in 2018 were used to (i) establish the invariance of the gender attitudes scale between countries and regions, over time, and (ii) study the evolution of gender attitudes from 2008 to 2018, at different levels of locality in Europe. Results showed that gender attitudes vary between regions (within the same country) even more than between countries, both in 2008 and in 2018, thus confirming the hypothesis that people’s attitudes towards gender and gendered roles are primarily attributable to local (socio-cultural) rather than national factors. The current paper contributes to the existing knowledge by (i) validating a measure of gender attitudes at different levels of locality that can be used in further studies; (ii) discussing the

methodological challenges related to the establishment of measurement invariance, also in a longitudinal perspective; and, (iii) providing empirical evidence about the importance of looking at locality in the study of gender attitudes to better inform policy,

practice and research.

09:15-

10:15

Awards

Session chair: Henrik Andersen

Best Paper by a Young Scholar: Participating in a panel survey changes respondents’

labour market behaviour. Authors: Ruben L. Bach and Stephanie Eckman

Panel survey participation can bring about unintended changes in respondents’

behaviour and/or their reporting of behaviour. Using administrative data linked to a

large panel survey, we analyse whether the survey brings about changes in respondents’ labour market behaviour. We estimate the causal effect of panel participation on the take-up of federal labour market programmes by using instrumental variables. Results show that panel survey participation leads to an increase in respondents’ take-up of these measures. These results suggest that panel survey participation not only affects the reporting of behaviour, as previous studies have demonstrated, but can also alter

respondents’ actual behaviour.

Best Paper: Spatial Regression Models: A Systematic Comparison of Different Model Specifications Using Monte Carlo Experiments. Author: Tobias Rüttenauer.

Spatial regression models provide the opportunity to analyze spatial data and spatial processes. Yet, several model specifications can be used, all assuming different types of spatial dependence. This study summarizes the most commonly used spatial regression models and offers a comparison of their performance by using Monte Carlo experiments. In contrast to previous simulations, this study evaluates the bias of the impacts rather than the regression coefficients and additionally provides results for situations with a nonspatial omitted variable bias. Results reveal that the most commonly used spatial autoregressive and spatial error specifications yield severe drawbacks. In contrast, spatial Durbin specifications (SDM and SDEM) and the simple spatial lag of X (SLX) provide accurate estimates of direct impacts even in the case of

misspecification. Regarding the indirect “spillover” effects, several—quite realistic— situations exist in which the SLX outperforms the more complex SDM and SDEM

specifications.

10:30-

12:00

Session 15:Issues in longitudinal data, panel data and secundary data

Session chair: Biagio Aragona

Presentation 1: Scaling Up Cross-National Surveys to a Global Societal Observatory –

the ONBound Data Cumulation and Linkage Project

Presentation 1 authors: Markus Quandt, Insa Bechert, Antonia May, Katharina Werhan, Annette Schnabel

Presentation 1 abstract:

Secondary analysis of social survey data is no recent invention. General social survey- type studies are often purposefully designed to be a public resource for the research community. With the advent of comparative survey programs such as the International Social Survey Programme, the European Values Study and the World Values Survey, and

‘Barometer’ surveys conducted in various regions of the world, this public-resource approach was taken to the level of cross-national comparisons already several decades ago. However, even the largest cross-national survey programs leave gaps in their coverage of countries and time-points. At this stage, the next level could be that of using ‘similar’ data from different survey programs to fill the coverage gaps that are left by each individual program. Combining data from different sources, however, is fraught with many challenges. Besides remaining coverage gaps, conceptual or methodological differences in the instrument design and data collection of these sources are likely to impair the comparability of the data, thus making their combination invalid. Further, many practical problems arise, such as intellectual property rights attached to the original data that may forbid re-distribution of derived data products. The presentation will introduce the strategies chosen and insights gained in the ONBound project (https://www.onbound.international/), which is cumulating survey data from across the world over many decades, and linking these data to contextual data. The topical focus of ONBound is on religion and national identity as markers of societal boundaries that may have changing significance across time and space. ONBound has to date compiled micro- level data for 1500 variables from 20 different survey programs conducted in 280 waves between 1970 and 2017, to be combined with about 4500 macro-level variables with a time frame starting in 1945. Resources from the project will be publicly available by the time of the presentation.

Presentation 2: Economic status and elections: A case study on voting behavior and economic hardship

in Rome

Presentation 2 authors: Gabriella D’Ambrosio, Giovanni Brancato, Marco Palmieri Presentation 2 abstract:

Several researches show the existence of a relationship between the electoral behavior of citizens and their economic condition. The so-called “economic voting theory” explains that, in periods of economic growth, citizens-voters reward the government considered the author of the well-being; on the contrary, in times of crisis, the population punishes it. The peculiarity of these studies, based on secondary data designed for purposes other than those of the scientific research, is to have a “national” territorial dimension, where the percentage of votes collected by the parties (both of government and opposition) in the political elections is related to the country’s macro- economic indicators (for example, GDP or unemployment rate). Nevertheless, up to now, only few scholars have analyzed this relationship at a “local” level due to the lack of statistical data on citizens’ economic conditions who live in territorially limited areas. This contribution presents the results of a study conducted in the city of Rome with the aim of understanding whether the electoral behavior of Roman citizens changes in function of the level of wealth/economic hardship in each of the 15 Municipalities of the Capital. To this end, data from two different databases are integrated. Indeed, Statistical Office of Rome provides national and local elections data (i.e., electoral consensus collected by both government and opposition parties); Italian Revenue Agency provides data on the economic conditions of families (i.e., number of families with minors and income of less than 25.000 Euros per year).

The results of this study offer empirical evidences in order to strengthen the correlation between voting and economic conditions.

Presentation 3: International students in higher education: the effect of student employment on academic performance

Presentation 3 author: Theresa Thies

Presentation 3 abstract:

An increasing amount of international students are immigrating to Germany to study at a German higher education institution. International students are mostly financed by parental support and student employment. In contrast to German students, they are rarely financed by governmental student loans. Previous research about the transition to paid student employment and its effect on academic outcomes is broad with different results. However, not much is known about the effect of student employment for international students. Furthermore, previous research struggles to account for the selection-into-employment mechanisms. Using hybrid models, this study explores whether student employment and different dimensions of employment (e.g., study- related employment, employment amounts) affect the semester grade point average (n=1,856) and the share of achieved credit points per semester (n=1,799). Using the first four semesters of a longitudinal study of international students at over 100 higher education institutions in Germany, the study shows that students with a higher employment tendency across waves have, on average, better semester grades than non- employed students (between-student-effect). When estimating the within-student- effect, it is demonstrated that changes to student employment and different employment dimensions do not change the semester grades. However, a change to employment and higher employment amounts decrease the share of achieved credit points per semester.

Presentation 4: Using address data enriched with additional information at regional level to identify less educated respondents before data collection

Presentation 4 authors: Uta Landrock, Christian Aßmann Presentation 4 abstract:

The German National Educational Panel Study NEPS collects data on educational processes and others. We find that less educated participants are underrepresented. This is true for the first wave of the panel survey and increases in subsequent waves. Therefore we want to investigate, whether itis possible to identify less educated participants before data collection. For this purpose we enrich the address data of the gross sample of our sub study with additional information at regional level (street section), using the infas360 database. We are currently developing an index to approximate the educational level of an address. Appropriate indicators maybe characteristics of the residential area (unemployment rate, density of buildings, etc.) types of building (living space per household), or purchasing power (income). The data will be collected in spring 2020. The target persons are 6-7 years old and will perform competence tests. Their parents will be interviewed via CAPI. The gross sample consists of 5,000 addresses, we aim to realize 1,000 interviews. The survey institute infas documents all contact attempts in order to identify three subgroups: non-contacts, refusals (which means successful contacts) and participants (realized interviews). We will compare the three subgroups to find out, whether less educated participants may be identified before data collection using additional information at regional level. We will use the collected information on the competencies of the children and on migration background and language spoken in the family, the (general and vocational) education and occupation of the parents as well as their income, collected in the parents’ survey.

In detail we will answer three questions: Does the index of education differ between the subgroups? Is the index predictive for the actual participation? And is it predictive for the measured competence level as well? If we can identify less educated target persons in advance, we could take this into account in our data collections.

Presentation 5: The Shadow of the Financial Crisis – Macro-level Socio-Economic Development and Feelings of Insecurity in Europe. A Between-Within Analysis of the European Social Survey, 2002-201

Presentation 5 authors: Lisa Natter, Dietrich Oberwittler Presentation 5 abstract:

Using all 9 waves of the European Social Survey merged with macro-level socio- economic and social policy indicators on the state-level, we employ hierarchical

“between-within” modeling to test assumptions about the detrimental effects of macro- level context factors on the feelings of insecurity in a longitudinal perspective, in particular against the backdrop of the Financial Crisis 2008 and its aftermath. Previous research (which however was mainly cross-sectional) has shown that the perceptions of insecurity are influenced less by crime levels and more by indicators of socio-economic disadvantage, and that welfare state stabilizers cushion these adverse effects. The

“between-within” design allows for the simultaneous analysis of variance between

European countries and of temporal variance within these countries. A special focus will

be on the appropriate modeling of temporal effects.

12:15-

13:45

Session 16: Special session Societal Complexity and Simulation

Session chair: Cor van Dijkum & Dorien DeTombe

Presentation 1: Corona crisis as a complex societal problem

Presentation 1 author: Cor van Dijkum Presentation 1 abstract:

Recently the world is confronted with an urgent complex societal problem: the fast spreading of a virus originated in China with severe consequences. It is a prototype of a complex societal problem the world will face in the future more and more. The question is how different actors handled this problem and how successful they were at last. From the framework of the methodology of handling complex societal problem such as is developed in the COMPRAM methodology one can set a methodological and scientific perspective on this question. We can look on open access data that available about the spreading of the disease with numbers for different countries expressing confirmed cases of contamination, people that became sick, deaths, recovered patients that became immune. But we also have to look at societal side-effects data showing: consequences in economy, the spread of information in social media about the disease, the way people react and organize themselves on the spread of disease. And we have to look at data showing the way different central institutions in our world, such as governments, non-governmental organizations, intergovernmental organization such as the WHO and scientific organizations handled the problem. To arrange, organize, explain, validate and understand all those data we develop a system dynamic simulation model originating from a (SIR) model of spreading of a disease in a population. With this model we will try to answer central questions: how successful this complex societal problem was handled, looking at health variables of people in the long run and other related social variables; how to explain this result and formulating new research questions. Thereby we compare two scenarios: (1) lockdown of a region or country such as was practiced in China, France and Spain; (2) a controlled delay of the spread of the disease to reach a group immunity of a region or country such as was practiced in the Netherlands.

Presentation 2: Coronavirus: a complex societal problem

Presentation 2 author: Dorien DeTombe

Presentation 2 abstract:

The coronavirus epidemic is a complex societal problem. The spread of the corona virus, also indicated as 2019-nCov or Covid-19, through the human population started in December 2019 in Wuhan South-East China caused by a contamination of a virus from a wild animal to a human being. This virus spread further from humans to humans. In three months time about more than 90.000 people were contaminated in China, which resulted to about 1900 deaths, about 2.2 % of the infected people, most of them elderly people which weak health.

Six weeks after the first signs of the virus the authorities of China start to take measures to prevent further contamination. The inhabitants of Wuhan were isolated from the world and from themselves. This has a huge impact on their personal lives.

The effect on the economy is also enormous. China is the supply warehouse, the magazine of the world. Because of the temporary closing of the factories many goods needed for the assembly lines of the factories in Western Europe and the USA could not be delivered on time, which resulted to a retarding of the production of many articles. The Corona epidemic is a complex societal problem. This issue should be handled according to the lines of the field of Methodology of Societal Complexity and the Compram Methodology in order to see which interventions can be taken and what the effect of the intervention on human lives and the economy is.

The Compram methodology handles the problem in a six step approach. Guided by a facilitator the problem will be analyzed, and possibilities for changes will be search, by first by experts and then by actors. The methodology emphasis knowledge acquisition and exchange of different experts and actors and takes into account the power and emotional differences.

Presentation 3: A simulation of swing voters’ dynamics

Presentation 3 author: Luciano Gallón Presentation 3 abstract:

An interesting complex social phenomenon that has existed since humans vote to make decisions, occurs when a voter changes his vote between one election and another. It may happen that he changes his vote for another candidate from the same party; by another candidate from another party; from voting to not voting; or from not voting to voting. This work presents a computational model that represents the situation, including a set of possible mechanisms to influence the voter to change his voting decision. The simulations of the model allow us to understand the magnitude of the situation as a percentage of voters who could change their vote in relation to the total number of voters. This proposal raises questions related to social research methodologies in which it is not easy to have control groups to validate hypotheses and questions statements related to the influence on the result of an election.

Presentation 4: SIMULATION OF COMPLEX SOCIETAL SYSTEMS. PROBLEMS, MODELS AND DATA SET REQUIREMENT

Presentation 4 author: Makarenko Alexander Presentation 4 abstract:

Recent investigations on complex societal systems require the applications of simulations. There exist the number of such models and approaches. The examples are multi-agent systems, differential equations, system dynamics, theory of networks, COMPRAM, OR and many others. But many problems still haven’t final solutions.

Especially it is valid for large social systems and for global processes. The example is the sustainable development problems.

The critical review of existing tools is proposed in the talk. Also some new models and

approaches of author are described (see [1-5]): models with associative memory, artificial life, models of individuals mentality, models with anticipation. Also the problems of special data base for such models are discussed. The role and importance of ontologies and cognitive maps in such models are described. Some examples of proposed approaches applications is given.

Presentation 5: Elite familiarity, class and political engagement: The case of Norway

Presentation 5 author: Jan Fredrik Hovden Presentation 1 abstract:

This paper draws on Pierre Bourdieu’s political and cultural sociology for a statistical investigation of the complex interrelationship between Norwegian citizens ́ varyingforms of attention, participation and attitudes towards the world of politics and public debate, how such engagement aligns and their homologies with social structures, focusing on classes and generations. Use multiple factor analysis it provides an uncustomary broad mapping of citizens use of media and culture, their participation in civil society and politics, and attitudes to news and politics. It identifies a first divide by citizens engagement and involvement in the worlds of social elites and a second by their forms and areas of political engagement. These roughly follows the dimensions of the underlying social space, and gender and generational divides. The results suggest the

continuing importance of Bourdieu’s homology thesis and informational capital for

understanding people ś political engagement.

14:00-

15:15

Session 17: Triangulation and Mixed Methods

Session chair: Susanne Vogl

Presentation 1: Assimilating interpretative and structural data: Merged Methods as an integrative approach of data collection

Presentation 1 author: Wander van der Vaart Presentation 1 abstract:

Research that focuses on complex social phenomena in society often requires information on both (aggregated) individual level and on the more structural level of meso or macro contexts that surround and influence individuals (cf: Coleman, 1986; Fulop, Maréchal & Rifkin, 2009). In this paper ‘merged methods’ is defined as a methodological approach that seeks to integrate both interpretative and structural components of such multi-level phenomena through one research instrument or

procedure. In ontological terms this means that the one main focus of data collection in a merged methods approach is precisely on the interaction between the different levels. In other words: it’s not only about picturing (aggregated) individual experiences on the one hand and higher level factors on the other, after which the researcher scrutinizes how these levels interact. It is precisely the interaction between different levels as such

– i.e., the inherent quality of the data that reflects the interaction between individual and higher-level information – that is the aim of data collection (Nico and Van der Vaart, 2012). A merged methods approach has very practical consequences for a study design, though the consequences may look pragmatic, they result from the essential core of

interdependency between ‘interpretative experience’ and ‘structural influences’. Such an approach demands methods that are largely tailor made for the phenomena at hand, which requires creativity and thoroughness from the side of the researcher. At the same time, precisely because the instrument or procedure needs to be tailored to specific situations, it may be vulnerable for bias. This paper discusses a rationale of merged methods in relation to practical consequences for research design. It is explored how a

merged methods approach may benefit from the assets of both interpretative and

structural approaches, while minimizing the weaknesses of each side.Key words: study approach, data collection, interpretative – structural, multi-level.

Presentation 2: Investigating Social Researchers‘ Methodological Beliefs with standardized Surveys. Results from Cognitive and Statistical Pretests Presentation 2 author: Felix Knappertsbusch

Presentation 2 abstract:

Methodological styles vary considerably between different areas of social research, but also within single disciplines such as sociology or educational science. It is commonly assumed that the methodological beliefs or mindsets of researchers play an important role in determining methods decisions and research design. This assumption is especially prominent in the mixed methods discourse on different research “paradigms”, which describes qualitative, quantitative and mixed research styles as based on different belief systems. However, the structure and quality of such belief systems is rarely investigated empirically (Bryman 2006, Sheehan/Johnson 2012). There is a small but growing

“prevalence rates literature” (Alise/Teddlie 2010) which analyses the frequency and quality of different research styles in various disciplines (Alise 2008, Truscott et al. 2010, Bryman 2006b). But this line of research almost exclusively relies on content analyses of published research and largely ignores the individual motives and beliefs “behind” the published “reconstructed logic”. Survey methods are one possible way to investigate methodological rationales and their underlying belief structures beyond the mere published methodological discourse. However, the issue of methodological beliefs and “paradigmatic” assumptions involves a number of abstract, difficult to define constructs, such as preferences for qualitative vs. quantitative methods, epistemological and ontological beliefs, discipline-specific social identities, etc. Thus, careful instrument development and testing is of special importance in developing survey-measures for these constructs. In the context of a study on the quality and motivation of mixed methods research practice, I am designing a survey instrument to measure methodological beliefs, partly building on previously existing measures (Sheehan/Johnson 2012; Williams et al. 2017). In my presentation I will present results from a qualitative (cognitive interviewing) as well as a standardized pretest of this instrument, and discuss the potentials and limitations of survey methods in this application. Alise, Mark A. (2008): Disciplinary differences in preferred research methods: a comparison of groups in the Biglan classification scheme. Dissertation, Louisiana State University. Online verfügbar unter https://digitalcommons.lsu.edu/gradschool_dissertations/2052, zuletzt geprüft am 05.11.2018. Alise, Mark A.; Teddlie, Charles (2010): A Continuation of the Paradigm Wars? Prevalence Rates of Methodological Approaches Across the Social/Behavioral Sciences. In: Journal of Mixed Methods Research 4 (2), S. 103–126. Bryman, Alan (2006b): Integrating quantitative and qualitative research. How is it done? In: Qualitative Research 6(1), S. 97–113. Sheehan, Michael D.; Johnson, R. Burke (2012): Philosophical and methodological beliefs of instructional design faculty and professionals. In: Education Tech Research Dev 60 (1), S. 131–153. Truscott, Diane M.; Swars, Susan; Smith, Stephanie; Thornton‐Reid, Flo; Zhao, Yali; Dooley, Caitlin et al. (2010): A cross‐ disciplinary examination of the prevalence of mixed methods in educational research 1995-2005. In: International Journal of Social Research Methodology 13 (4), S. 317–328

Presentation 3: Using mixed methods approaches in validating psychometric instruments

Presentation 3 authors: Florian Berens, Kelly Findley Presentation 3 abstract:

In the development of psychometric instruments, two core activities for validation have traditionally played a major role: (Think Aloud) Cognitive Interviews and Explorative or Confirmative Factor Analysis. Cognitive interviews may be used to check the comprehensibility and formulation validity of individual item wordings. Factor analysis quantitatively tests the reliability of the theoretically designed constructs. From the authors’ point of view, this procedure lacks an intermediate stage that using mixed method approaches that combines a more detailed examination of the individual items with a check of the validity of the constructs. The idea behind our approach is to substitute the formation of factors via correlation measures—common in quantitative factor analysis—by human clustering. Interviewees would cluster the developed items into factors based on their personal knowledge and their everyday understanding of the questionnaire. The required think aloud and the result of the clustering can be used to evaluate and improve item formulations and intended constructs. The amount of preliminary information the respondents receive can be varied. This paper reports on the authors’ experiences with different variants of the approach, which we tested on an instrument to assess students’ beliefs about statistics. All variants were tested at two universities in different countries on different groups of respondents. The result is a procedural proposal with which an instrument draft can be substantially improved within two cycles. In the first cycle, items and theoretical constructs are compared

individually to check their fit. In the second cycle, the constructs as such are recreated by the respondents.

Presentation 4: Mixed Methods Research Citations Network: Communities of Scholars and the Production/Dissemination of Academic Knowledge

Presentation 4 authors: Noemi Novello, Alessandra Decataldo Presentation 4 abstract:

Within the field of mixed methods research (MMR), different approaches to integration coexist: many are the theoretical and epistemological standpoints, intertwined in a rather complex panorama. Through a citation network analysis (CNA), we aimed to identify and observe different communities of scholars within the MMR field in social science. CNA represents an effective strategy to explore processes of knowledge production and dissemination. Publishing papers in academic journals is a part of the evaluation system in the academic world and citations are a way to acknowledge papers and their authors. Therefore, identifying the most cited authors and communities of scholars quoting each other thanks to the strategy of CNA, we can analyze acknowledged scholars who are part of a community and thus refer to the same perspective in MMR. Moreover, we are interested to look at how groups with different approaches interact and communicate one each other.

A first endeavor of this study was part of a larger dissertation on MMR. We then decided to update the analysis and we found out that the number of eligible records was enormously increased – most likely due to the increase of indexed journals. Specifically, we conducted a CNA on a set of papers published in English in academic journals, starting from 2003, year when the Handbook of Mixed Methods in Social & Behavioral Research by Tashakkori and Teddlie was published. We worked on data extracted from the citation-based database Scopus (Elsevier), selecting those papers that 1. refer to the large area of social science; 2. explicitly make use of MMR; 3. are empirical studies with primary data; 4. are available in digital format. Analyzing citation networks made by first authors (node) and citations (edges), we are then able to focus on various groups of

scholars and relationships among them to describe the MMR field.

15:30-

17:00

Session 18: Special issues on data collection in survey research

Session chair: Tobias Gummer

Presentation 1: Using fictitious issues to investigate cognitive processes in surveys Presentation 1 authors: Henrik Andersen, Jochen Mayerl, Felix Wolter, Justus Junkermann

Presentation 1 abstract:

Satisficing theory suggests that motivated respondents take their time to respond carefully and provide high quality data, while unmotivated respondents rush through and answer without much thought, thus providing poor quality data. Based on this, survey researchers often look at survey response latencies, i.e., the amount of time taken to answer a survey question, as an indicator of data quality. Fast responses are taken as an indicator of satisficing behaviour, while slow responses can be taken to indicate optimizing. However, some forms of dual process theory, like the Motivation and Opportunity as Determinants (MODE) model posit that strong, salient attitudes can be expressed spontaneously, without much deliberation. This poses a problem for the application of response latencies to assess the quality of responses because fast responses can mean that the respondent was weakly motivated, or that they held a strong opinion and were able to express it quickly. Obviously, responses based on strong salient attitudes are more desirable for the survey researcher than answering heuristics based on satisficing. We propose using fictitious issues, i.e., questions about nonexistent or highly obscure topics to investigate how respondents in surveys tend to approach unfamiliar topics. This approach excludes the possibility of chronically accessible (“strong”) judgments and makes it easier to interpret response latencies. In the context of fictitious issues, deliberate substantive responses, so-called ‘pseudo-opinions’, are a sign of a respondent attempting to form an ad hoc judgment, while fast substantive responses can be taken as a sign that the respondent used some heuristic short-cut. We find a nonlinear effect of response latencies on the probability for respondents to express pseudo-opinions. The more quickly a respondent answers, the more likely they provide a pseudo-opinion, suggesting satisficing is at play. But the probability of a pseudo-opinion increases again for slower responses, suggesting some motived respondents attempted to decipher a meaning to our fictitious issues and answer accordingly. We use the results to make generalizations about how respondents react to unfamiliar topics and assess the extent of satisficing behaviour in our survey. We also look at other correlates to investigate whether, say, social desirability bias influences the likelihood of pseudo-opinions.

Presentation 2: Capturing the interaction between question order effects and visual layout: results from an online experiment

Presentation 2 authors: Ádám Stefkovics, Zoltán Kmetty, Koltai Júlia Anna Presentation 2 abstract:

Question order effect refers to the phenomenon that previous questions may affect the cognitive response process and respondents’ answers. Previous questions generate a context or frame in which questions are interpreted. At the same time, in self- administered surveys, respondents are required to visually process the questions, therefore visual design may also shift responses. Past empirical research has yielded considerable evidence supporting the influence of question order, but few studies have investigated how question order effects interact with visual design. To address this

research gap, the present study uses data from an online survey experiment conducted on a non-probability-based online panel in Hungary, in 2019. We used the welfare-

related questions of the 8’th wave of ESS. Respondents were asked about the perceived consequences of social benefits and services (E10, dependent variable). We manipulated the questionnaire by changing the position of a question that calls forth negative stereotypes on such social benefits and services (E13). One group received E13 in its original place (after E10, control group), one group received E13 just before E10 and another group received E13 before E10, but with one question between the two (E9).

We further manipulated the visual design by presenting the questions on one page (grid) or in separate pages. This resulted in a 3X2 design. 1100 respondents were randomly assigned to one of the six groups. We hypothesized that placing E13 right before E10 will shift responses and the effect will be stronger if questions are presented on the same page. The results show that placing E13 right before E10 significantly changed

respondents’ attitudes in a negative way, but the effect is significant only when questions are presented on separate pages. A possible interpretation of the results is that such one-page per question design leads to a deeper cognition of the questions.

Presentation 3: Possibilities and Limits of the Joint Modeling Approach for Biosocial Empirical Analysis

Presentation 3 authors: Alessandro Procopio, Robin Samuel Presentation 3 abstract:

The increasing number of biosocial surveys, which integrate biomarker measurements into social surveys, has been enabling the inclusion of biological data into social research. In this paper, we assess the Joint Modeling approach as a statistical tool to explore both biological and social data in longitudinal biosocial surveys. We argue that this model can be of great use for demographic and sociological research with a focus on health inequalities, as many research questions in those fields that involve time-to-event analysis. This kind of analysis, however, is prone to problems of unobserved heterogeneity and time misspecification. We assess how the Joint Modeling approach fares under these conditions using Monte Carlo Simulation. The results suggest that the model is fairly robust to these issues. We further conclude to put more attention on a theory-driven construction of the Joint Models, as this will allow to fully benefit from their capability in biosocial empirical analyses.

Presentation 4: Response behavior and quality of survey data: Comparing elderly respondents in institutions and private households

Presentation 4 author: Jan-Lucas Schanze Presentation 4 abstract:

An increasing age of respondents and cognitive impairment are usual suspects for increasing difficulties in survey interviews and a decreasing data quality. This is why survey researchers tend to label residents in retirement and nursing homes as hard-to- interview and exclude them from most social surveys. In this article, I examine to what extent this label is justified and whether quality of data collected among residents in institutions for the elderly really differs from data collected within private households. For this purpose, I analyze the response behavior and quality indicators in three waves of Survey of Health, Ageing and Retirement in Europe. To control for confounding variables, I use propensity score matching to identify respondents in private households who share similar characteristics with institutionalized residents. My results confirm that

most indicators of response behavior and data quality are worse in institutions compared to private households. However, when controlling for sociodemographic and

health-related variables, differences get very small. These results suggest the importance of health for the data quality irrespective of the housing situation.

Presentation 5: Integrating Probability and Nonprobability Samples for Survey Inference

Presentation 5 authors: Joseph W. Sakshaug, Arkadiusz Wiśniowski

Presentation 5 abstract:

Survey data collection costs have risen to a point where many survey researchers and polling companies are abandoning large, expensive probability-based samples in favor of less expensive nonprobability samples. The empirical literature suggests this strategy may be suboptimal for multiple reasons, among them that probability samples tend to outperform nonprobability samples on accuracy when assessed against population benchmarks. However, nonprobability samples are often preferred due to convenience and costs. Instead of forgoing probability sampling entirely, we propose a method of combining both probability and nonprobability samples in a way that exploits their strengths to overcome their weaknesses within a Bayesian inferential framework. By using simulated data, we evaluate supplementing inferences based on small probability samples with prior distributions derived from nonprobability data. We demonstrate that informative priors based on nonprobability data can lead to reductions in variances and mean squared errors for linear model coefficients. The method is also illustrated with actual probability and nonprobability survey data. A discussion of these findings, their implications for survey practice, and possible research extensions are provided in

conclusion.

17:15-

18:45

Session 19: Challenges in Establishing Validity of Measurements in the Age of Digitalization and Globalization

Session chair: Natalja Menold, Robert Neumann, Peter Graeff

Presentation 1: How Do Reverse-Keyed Items in Inventories Affect Measurement Quality and Information Processing

Presentation 1 author: Natalja Menold

Presentation 1 Abstract: The common practice of using reverse-keyed items is addressed. Reverse-keying means that items in multi-item inventories (or scales) are differently associated with the underlying concept. The proponents of reverse-keying argue that it helps to minimize, prevent or control for an acquiescence bias or for non- attending, careless response behavior. Alternatively, confirmation bias can explain why respondents provide similar responses to reversals. It occurs because respondents activate beliefs depending on how an item is stated. Apart from the biased responses, there are other explanations for why study participants respond similarly to reversals. Since it can be difficult to find polar opposites, scale developers might assume that reversals present opposite meanings for the same content, while respondents may assume such items have different content and agree to both items due to the corresponding true responses. Likewise, the ability to manage reversals can be limited, as it was found to be correlated with cognitive abilities.

In randomized experiments, inventories with reverse-keyed items are compared with inventories in which all the items are either positively or negatively associated with the underlying concept. The results show that with reverse-keying, a control of the potential bias was not sufficient, as well as the factorial structure, reliability, and validity were negatively affected. An eye tracking study revealed that respondents did not process

information more deeply with the reverse-keyed items than with the other forms.

Respondents seemed to find it difficult to process reverse-keying mentally so that it is not sufficient to use it in heterogeneous samples and with short inventories.

Presentation 2: Cybercrime measurement and issues of validity: Comparing process- generated and survey data in Europe

Presentation 2 author: Julia Kleinewiese

Presentation 2 abstract: Typically, measurements of crime have struggled with issues of validity (e.g. Skogan 1974). Since some crimes are reported, while others are not, the degree of validity depends on the amount of crimes remaining in the dark.

Validity problems frequently occur when cybercrime is scrutinized. This deviant phenomenon has gained increasing attention as technologies have advanced and the use of digital devices and computer-based systems has become widespread. Cybercrime is being examined by a rising number of researchers but issues pertaining its valid measurement have not yet been a central focus of such research (see Kshetri [2013] for an introduction to problems of cybercrime measurement). This leads to the question: If a good deal of cybercrime-activities remain in the dark (net), how can the validity of cybercrime data be improved? My suggestion is that a combination of different data sources could add to improving data validity. For this purpose, a focus is placed on process-generated and survey data in Europe. In my presentation, I will compare different data sources, such as data from surveys and administrative institutions (also big data), with reference to their particular contributions in order to bring light into the dark of unreported cybercrimes. Here, the official figures work as a benchmark and bottom-line for discovered and reported crimes. Moreover, they are based on clear (legal) definitions that leave out several relevant issues. Therefore, surveys may add to the understanding of cybercrime, regarding both the frequency of occurrence and phenomena outside of legal notions.

Presentation 3: Are you concerned yet? Measuring privacy concerns Presentation 3 author: Oliver Brust, Robert Neumann, Hagen von Hermanni Presentation abstract: In the current age of information, our daily actions have undergone a substantial change with regards to privacy. Activities that were once

performed in private now regularly leave traces of data whenever and wherever we are. Whether searching for a job, traveling and commuting, dating, connecting and communicating with friends and relatives, streaming videos or during physical activities: Across all activities, preferences can be recorded and predicted, defaults are documented, geolocations are tracked and individuals are profiled. Despite the possible advantages for individuals and companies in being monitored or monitoring constantly, substantial concerns about data security and privacy arise. Current research investigates the level of privacy concerns, the consistency of privacy attitudes and how both are linked to actual behavior. Yet investigating these matters (e.g. the privacy paradox) requires a valid measurement of privacy concerns which usually varies with the topic at hand. Although privacy scales developed by Westin have often been criticized, we consider them a starting point to our goal of establishing a generalized short scale on privacy concerns. We assess an improved privacy scale using data from two separate surveys of students as well as a general population sample respectively. Besides investigating the scale properties, we also present preliminary results of its explanatory power in regard to self-reported behavior (incl. results from factorial survey designs).

Presentation 4: What determines students’ willingness to share data? Evidence from a

Factorial Survey Experiment

Presentation 4 authors: Edgar Treischl, Tobias Wolbring

Presentation Abstract: Given its ability to handle a large amount of data, artificial intelligence (AI) has the potential to improve data-driven decisions under various situations. The present research identifies the necessary conditions for the implementation of an AI-based advisory system (AS) in higher education. Using a factorial survey design, we examine experimentally varied features of an AI-based AS to explore students’ willingness to use it and students’ willingness to share their data as a core challenge for successful implementation. Theoretically, we focus on the perceived costs and benefits to explain students’ intention, but we also highlight the role of trust and privacy concerns in regard to collecting data for the AS. In terms of benefits, information about the predictive power of the AS significantly increases students’ intention to use the tool and to share data and thus offers an incentive for students to share data. Moreover, a disproportionately long survey duration and survey topics that seem unrelated to the AS reduce students’ willingness to share data. With respect to trust and privacy concerns, our results indicate that providing transparent information about the AS has no effect on students’ willingness to share data, while aspects regarding who has access to the AS results and a long period of data storage reduce students’ intentions to share data. Based on these findings, we advise universities to

communicate students’ expected benefits from a system to implement the AS, but we also recommend seriously considering students’ privacy concerns. Who has access to the data and the results of the AS should be transparent, as well as for what reason and how long. Otherwise, a substantial and probably selective part of the student body may not use the tool or share data due to privacy concerns.

Presentation 5: Are measures of democracy valid in all circumstances?

Presentation 5 authors: Claire Durand et al.

Presentation Abstract : This paper focusses on the different attempts at measuring the level of democracy in all the countries of the world. There are a number of measures of “democracy”. The most used are the Freedom House index, Polity IV, the Economist’s index, V_Dem (Varieties of Democracy) and the Global State of Democracy. These measures may start from a broader or a more restricted conception of democracy. The question is how to assess their validity while taking into account the large variety of contexts in different countries. We first examine the content validity of the various measures and show what conceptions of democracy they refer to. We then go on to assess construct validity, that is, whether the different subscales do indeed measure different concepts and we examine predictive validity, that is, whether measures of democracy and their various subscales relate to relevant external variables. We argue that since democracy is defined primarily as a “government by the people for the people”, the more democratic a country is supposed to be, the more people should to trust their institutions. Finally, we aim at identifying country characteristics that explain the relationship between trust and measures of democracy. The data used to answer these questions are 1) a database of more than 20 million measures of institutional trust answered by 2 million respondents from 143 countries, mostly outside the western world, over 21 years. 2) a database of democracy indices for these same countries and period 3) a database of each country’s economic, social and political characteristics. We conclude on the possible bias of democracy indices when applied to very diverse situations and historical contexts at the country level and on the methods that could and

should be mobilised in order to assess validity in such circumstances.

18:45-

19:00

Closing Remarks