Welcome, Guest
Username Password: Remember me

TF2 (WG1) Evaluation of questionnaire quality
(1 viewing) (1) Guest
  • Page:
  • 1
  • 2

TOPIC: TF2 (WG1) Evaluation of questionnaire quality

TF2(WG1): Evaluation of questionnaire quality 4 years, 8 months ago #130

Task force 2: Evaluation of questionnaire quality

WG1 Measurement and non-response errors resulting from web survey questionnaire design.

Task force leader: Ana Slavec
Task force members: open to all, please join.

Aim/Central research question:
This TF deals with the quality of survey data collected by uni-mode web surveys where only web questionnaire is used to gather survey data. Mixed-mode surveys including a web component are addressed in TF3. Here, we are mainly focusing on measurement errors and item-nonresponse resulting from the design of the questionnaire while other errors (such as unit-nonresponse, coverage, sampling, mode effect etc.) are not addressed by this TF.

There are several activities that can be organized within this TF. WebDataNet members are encouraged to propose activities (joint experiments, papers etc.) that would fit into this topic.

As a first activity we propose to prepare a list of criteria that can be used to evaluate the quality of a web survey questionnaire. When discussing quality, we will focus on measurement errors and non-response (particulary item-nonresponse) errors that results directly from the design of the web survey questionnaire. We would like to apply this list within the free of charge online tool for implementing web surveys 1ka.si that is developed at the Centre for Social Informatics at the Faculty of Social Sciences, University of Ljubljana. This tool already includes a set of criteria which gives a user (the researcher who is setting up a survey) an evaluation of the quality of the questionnaire and problems detected. We would like to take the advantage of WebDataNet to involve experts on web survey methodology from the WebDataNet network (and maybe some external experts) to comment and improve the set of existing criteria.

We ask WebDataNe members to propose other possible activities.

Organisation/planning:
At the Ljubljana 2012 November meeting we would like to organize a 2 – 3 hours working meeting of the WebDataNet experts on web survey methodology (especially web questionnaire design). After presenting the TF and collecting ideas for different activities, we would start with the first activity of preparing a list of criteria to evaluate web questionnaire quality. During the meeting we would present the 1ka application, the criteria for questionnaire evaluation and organize a structured discussion of experts on this topic. A result would be an improved set of criteria for questionnaire evaluation to include in 1ka.si.

Planned outcome:
Although we would like to have an improved set of criteria already after the next meeting in November, this task force could continue its work. For the next meeting (Mannheim, spring 2013) we (the 1ka.si developers) could prepare en evaluation of the improved set of criteria (how it works in practice) and invite experts again to discuss and eventually improve it. This could be continued to the next meetings.
Katja Lozar Manfreda
Chair of WG1: Quality issues in Web surveys

Katja Lozar Manfreda, Phd
Assistant professor
University of Ljubljana, Faculty of Social Sciences
Kardeljeva pl. 5, 1000 Ljubljana, Slovenia
Tel: +386 1 5805 313
Email: This e-mail address is being protected from spambots. You need JavaScript enabled to view it -lj.si
Last Edit: 4 years, 7 months ago by Lozar Manfreda.

Re: TF2(WG1): Evaluation of questionnaire quality 4 years, 5 months ago #257

  • aslavec
  • OFFLINE
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
Summary of discussions and presentation slides used at meeting:

Last Edit: 4 years, 5 months ago by aslavec. Reason: attachments did not upload

TF2 (WG1) Evaluation of questionnaire quality 4 years, 4 months ago #266

  • aslavec
  • OFFLINE
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
Task force 2: Evaluation of questionnaire quality

WG1 Measurement and non-response errors resulting from web survey questionnaire design.

Task force leader: Ana Slavec
Task force members/attended previous meeting: Marek Fuchs, Reuven Dar, Goran Milas, Vaka Vésteinsdóttir, Gudbjorg Andrea Jonsdottir, Silvia Biffignandi, Fanney Thorsdottir, Kirstine Kolsrud, Annette Scheprenzel, Konstantinos Tsagarakis, Knut Kalgraff Skjak, Kea Tijdens, Ana Slavec.
Open for all, please join.

Aim/Central research question:
This TF deals with the quality of survey data collected by uni-mode web surveys where only web questionnaire is used to gather survey data. Mixed-mode surveys including a web component are addressed in TF3. Here, we are mainly focusing on measurement errors and item-nonresponse resulting from the design of the questionnaire while other errors (such as unit-nonresponse, coverage, sampling, mode effect etc.) are not addressed by this TF.

There are several activities that can be organized within this TF. WebDataNet members are encouraged to propose activities (joint experiments, papers etc.) that would fit into this topic.

As a first activity we propose to prepare a list of criteria that can be used to evaluate the quality of a web survey questionnaire. When discussing quality, we will focus on measurement errors and non-response (particulary item-nonresponse) errors that results directly from the design of the web survey questionnaire. We would like to apply this list within the free of charge online tool for implementing web surveys 1ka.si that is developed at the Centre for Social Informatics at the Faculty of Social Sciences, University of Ljubljana. This tool already includes a set of criteria which gives a user (the researcher who is setting up a survey) an evaluation of the quality of the questionnaire and problems detected. We would like to take the advantage of WebDataNet to involve experts on web survey methodology from the WebDataNet network (and maybe some external experts) to comment and improve the set of existing criteria.

We ask WebDataNe members to propose other possible activities.

Previous meeting:
At the meeting in Ljubljana in November 2012 November meeting we organized a working meeting of the WebDataNet experts on web survey methodology. The summary can be accessed at mi.ris.org/uploadi/editor/DnD1358777762TF2.docx

Organization/planning:
For the next meeting (Mannheim, March 2013) we (the 1ka.si developers) will prepare en evaluation of the improved set of criteria and will invite experts again to discuss and eventually improve it. This could be continued to the next meetings.
In addition, a discussion of different desings using a discoursive approach can be organised.
Also suggestions of activites from other members are very welcome.
Last Edit: 4 years, 4 months ago by aslavec.

Re: TF2 (WG1) Evaluation of questionnaire quality 4 years, 3 months ago #290

  • umatzat
  • OFFLINE
  • Fresh Boarder
  • Posts: 2
  • Karma: 0
Dear Ana (and others),

I'm in a group of researchers at Eindhoven University who are studying quality of data in web surveys that aim at measuring respondents' social networks (ego-centered networks). We are trying out different ways of enhancing this type of web surveys. We are experimenting with videos and with a new tool that uses visualizing methods for eliciting respondents' reactions to network items.

I am considering joining TF2 and would be curious to hear whether this fits with the goals of TF2. Also, I would be curious to hear whether others have an interest in measurement of social netowrks in web surveys.

I will be available at the GOR conference and also at the Webdatanet meeting in Mannheim. So should anybody have an interest in talking or collaborating, please approach me face-to-face (or by email).

Thank you,
Uwe Matzat

Re: TF2 (WG1) Evaluation of questionnaire quality 4 years, 3 months ago #295

  • aslavec
  • OFFLINE
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
Dear Uwe,

thank you for your interest, your research would fit very well in this TF. You are very welcome to the TF meeting on Friday at 9:30. I see you have a presentation on Thursday morning - if you would like to present something also on Friday please let me know. My e-mail is ana.slavec (at) fdv.uni-lj.si.

By the way, a while ago some that Katja Lozar Manfreda and some other researchers at our faculty have done some research on measuring ego-centered network online. I thought you might be interested in their papers:
mrvar.fdv.uni-lj.si/pub/mz/mz1.1/lozar.pdf
www.stat-d.si/mz/mz5.1/abst/kogovsek.htm

Looking forward to meet you,
Ana

Re: TF2 (WG1) Evaluation of questionnaire quality 4 years, 3 months ago #306

  • mernstst
  • OFFLINE
  • Fresh Boarder
  • Posts: 9
  • Karma: 0
Dear Ana and other TF2 members,

Gerry just posted a reference in TF3, which could be interesting for this TF:
It is a guidance on how to design questions for Mixed mode surveys, including web. This could give some additional references. But also, I think integrating recommandations concerning the mix of modes could be a pertinent thing.

best Michèle



Copy of her message:

Over a year ago, Pam Campanelli and I (with others) produced guidance on designing questions for mixed mode surveys. The guidance is structured by the type and format of the question, and is based on evidence from the literature. We would like to share it more widely but it is unclear how. It's current format is not really suitable for a publication. We would welcome any thoughts that you may have about the format, content and possible method of sharing this resource. You can find the guidance at

surveynet.ac.uk/sdmi/events.asp

It (handout.pdf) is listed under the event "11 October 2011 - Mode Effects on Survey Measurement".
Dr. Michèle Ernst Stähli
Senior Researcher
FORS (Swiss Centre of Expertise in the Social Sciences)
c/o University of Lausanne, Vidy
CH-1015 Lausanne
The following user(s) said Thank You: aslavec
  • Page:
  • 1
  • 2
Time to create page: 0.21 seconds