A Probability-based Web Panel in Taiwan
Due to the advantages of lower cost and higher efficiency, online surveys have been more commonly used than other survey modes. The disadvantages of volunteer-based or non-probability online surveys, however, are associated with lack of a sampling frame, selection bias, and coverage issues which often result in lower data quality and skewed response distributions. Recent efforts have been made to made to use probability-based online samples of the target population to achieve representativeness and quality data for inference. While most of the probability-based web panels have been initiated in Europe, a growing number have been used in other continents.
In Taiwan, a probability-based web panel has been established by the Center for Survey Research (CSR) at the Research Center for Humanities and Social Sciences, Academia Sinica. Starting from 2012, CSR collected email addresses from respondents who have participated in surveys and agreed to provide their email addresses for future web surveys conducted by CSR. The offline population is not included in the web panel. The CSR web panel was officially established after approval by the Institutional Review Board in 2016. The sources of emails include those obtained from CAPIs since 2015 and those from CATIs since early 2016. As is usual for face-to-face or CAPI interviews, stratified multi-stage PPS sampling method has been applied. For CATIs, CSR uses random digit dialing (RDD) using stratified multi-stage probability proportional to size (PPS) to select respondents aged 18 years and older in Taiwan.
With regard to the increasing cell-only population, CSR has employed different sampling strategies for CATIs and online surveys to obtain probability samples. This also increases the channels for the recruitment of the web panel, as well as the coverage of the target population for online surveys in Taiwan. A dual-frame sampling approach for the selection of both landlines and cellphone users is adopted for CATIs in 2019. For online surveys, a hyper link is embedded in text messages for cell-phone users who are contacted using a RDD-PPS approach in 2020. The procedure for obtaining the respondents’ willingness and agreement to participate in future web surveys remains the same. The availability of email addresses is no longer essential for future contacts, if the respondent can provide an active cellphone number.
After the collection of email addresses from CAPIs, CATIs or online surveys, invitation to participate in the CSR web panels is sent to those addresses. The acceptance rate has increased in the past two years, from about 30% to about 35%, and the number of new members of the web panel is about 2,000 and 3,000, respectively. As of the end of November 2021, the number of members in the CSR web panel is 8,141, with a completion rate ranging from 45% to 50% of them are active panel members. The web panel has a slightly higher proportion of females than that of males. Similar to other probability-based web panels, the CSR panel over-represents younger and more highly educated persons, and under-represents older persons. The majority of the panel members are between 18 to 49 years old, with 24% being aged 18-29 years old, 26% aged 30-39 years old, 25% between 40-49 years old and 16% aged 50-59 years old. About half of the panel members have a college degree and 22% of them have attended graduate school.
The CSR web panel manages the online survey operation mostly in house, using their multi-mode computer-assisted interviewing system (multi-mode CAI system) for the Windows and Android operating systems. This includes the submission of questionnaires, programming and testing of the online questionnaires, and the management of panel members. Panel members receive incentives, which are usually electronic cash vouchers for convenience stores, for each survey they have completed.
The CSR panel is a general survey research infrastructure that allows researchers to submit proposals for data collection in the panels on a pay-per-survey basis. Researchers are also encouraged to archive the survey data in the Survey Research Data Archive (SRDA) of CSR, where data are available free of charge via membership. In 2021, eight online surveys have been conducted using the CSR web panel. The length of interviews ranges from 10 to 15 minutes. The completion rate of the most recent survey on the CSR’s web panel is 46%. The recruitment of this probability-based web panel continues, so we expect the number will keep growing.
Special thanks to the CSR staff, especially Fang-Yu Su and Siou-huei Su.
By Pei-shan Liao (RC33 vice-president at large for East Asia)