Welcome, Guest
Username Password: Remember me

TF20 (WG2) Web Survey Paradata
(1 viewing) (1) Guest
  • Page:
  • 1
  • 2

TOPIC: TF20 (WG2) Web Survey Paradata

TF20 (WG2) Web Survey Paradata 3 years, 10 months ago #358

Attention: Paradata can be collected with many survey modes. This TF is focused on web survey paradata.

Task force leader: Ioannis Andreadis
Task force members: open to all, please join.

Aim/ Research questions:
Mick Couper invented the term paradata in a presentation at the Joint Statistical Meeting in Dallas to describe process data produced by a computer assisted survey information collection system. Paradata provide information on how respondents interact with a web questionnaire. Respondents' actions can be captured through their keystrokes, mouse movements and clicks and by studying their navigation around the pages of the questionnaire. This can be done without additional costs because paradata can be automatically generated by the web survey software.

Types of Paradata in Web Surveys
Web Survey Paradata can be Server-side or Client-side.

Server-side paradata are data collected by page requests on the web server which are recorded on log files. The interested researcher can extract the following information: IP address, date and time, method, requested page, referring page and user-agent. From the previous page, requested page and the corresponding times we can observe the path of pages followed by the user and the time spent on each of them. From the user-agent string we can learn the browser, the operating system and the device used by the respondent.

Client-side paradata can provide more detailed information but this method works only if the user has not disabled JavaScript. Another issue with JavaScript is that it requires some time to load and in some cases the user finishes some actions before the script is fully loaded; as a result JavaScript fails to capture these user actions. On the other hand, there are a lot of benefits with client-side paradata: Response times can be recorded for each item instead of the page level. In addition, we can track changed answers, mouse movements and mouse clicks. Finally, we can have much more information about the browser (e.g. screen size, window size, etc).

Why are paradata important? How can we use them?
Paradata can be used for two tasks: i) post-survey quality assessment of the online questionnaire and ii) interactive adaptation of the web survey according to the respondent's behaviour

Post-survey quality assessment:
Paradata can provide valuable information about the process of the web survey, which can be used to identify problems regarding the layout or the format of response options in order to improve the quality of the survey. Observing how respondents interact with the web survey may help us make it less burdensome for them. For instance, if the number of changed answers to a question is significantly larger than the corresponding number of changes for other questions, this could mean that the specific question is problematic. Much longer response times, frequent use of supplementary definitions and help files could be a sign of a question that is not clear and should be rephrased.

Interactive adaptation:
Paradata can be used to identify respondents with unusual behaviour that should trigger the appearance of special targeted messages to them. For instance, if some respondents answer the questions too fast, we can display a message encouraging them to pay more attention to the questions.

The main aim of this TF is to bring to together researchers who have collected and/or want to work on Web Survey Paradata in order to facilitate the comparative analysis of the available datasets.

Who should join the Task Force?
Have you collected web survey paradata?
Are you planning to collect web survey paradata?
Do you want to analyse web survey paradata?
Are you interested in a joint publication based on the analysis of web survey paradata?

If you have answered "Yes" to at least one of the above questions, please reply to this post and express your intention to join the task force.

Organisation/planning:

The Web Survey Paradata Task Force will meet in Reykjavik. If you have already collected paradata, please bring with you a list of the types of Web survey paradata you have in your hands. During the Reykjavik meeting we will try to identify the possibilities of co-authored publications and joint research projects.

Planned outcome:
  • The preparation of co-authored publications. Probably a special journal issue web survey paradata.
  • Joint research projects on web survey paradata


Andreadis, Ioannis. Paradata from Political Web Surveys. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2014-09-26. doi.org/10.3886/E17816V1
Last Edit: 2 years, 9 months ago by Ioannis Andreadis.
The following user(s) said Thank You: vogel

Re: TF20 (WG2) Web Survey Paradata 3 years, 9 months ago #372

  • vogel
  • OFFLINE
  • Fresh Boarder
  • Posts: 17
  • Karma: 0
Hi.

Yes, I'd be pleased to chat about this in Reykjavik and beyond. I'm interested in various aspects of linguistic anomaly, and where the paradata includes time-stamping and edits of text fields, that are
otherwise interpreted in their final form, I could see interesting collaborations emerging.

All my best,
Carl
The following user(s) said Thank You: Ioannis Andreadis

Re: TF20 (WG2) Web Survey Paradata 3 years, 9 months ago #374

Dear Carl,

I find your idea of studying the relationship between linguistic anomalies and paradata very interesting. I am looking forward to discussing the details during the TF20 meeting. According to the agenda it will take place on Tuesday 17 September at 16:00.

Kind regards,
Ioannis

Re: TF20 (WG2) Web Survey Paradata 3 years, 9 months ago #375

Thanks a lot Ioannis, I may not be able to join this meeting (TF17) but please keep me informed.
Your meeting is also taking place at the same time that the master one. You may ask Alberto to inform you about sevarl imporatnat issues that will be discussed.

Best
Pablo

Re: TF20 (WG2) Web Survey Paradata 3 years, 9 months ago #383

  • aslavec
  • OFFLINE
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
Hi all, at the TF meeting I mentioned I attended a presentation on mouse movements paradata by Rachel Horwitz, student at JPSM (University of Maryland). Here is the link on her dissertation on this topic:
drum.lib.umd.edu/bitstream/1903/14039/1/..._umd_0117E_14107.pdf

Re: TF20 (WG2) Web Survey Paradata 2 years, 8 months ago #409

  • Matjašič
  • OFFLINE
  • Fresh Boarder
  • Posts: 1
  • Karma: 0
Greetings,

I am one of the participants of Webdatanet 2nd Training School: Paradata. As I am very interested in paradata I would be glad to join paradata task force.

Sincerely,

Miha Matjašič.
  • Page:
  • 1
  • 2
Time to create page: 0.21 seconds