Survey Design for Citizens’ Panels

Citizens’ panels are an easy and cost-effective way for local government to consult with their community. A lot of thought and planning needs to go into a survey even before the first question is drafted. Surveys take a lot of time and resources to develop, analyse and report on and you do not want to get to the drafting of the final report and find that the survey data does not actually supply you with the information you require or that the data is unusable, as they say, prior planning and preparation prevents poor performance. This article will go over some of the best practice when designing surveys for panels.

Citizens’ panel surveys often stem from a piece of work or a project being carried out by a local government, so it is important to keep in mind what the objective of the overall project or work is but also to develop an objective for the survey itself. It is a good idea to include the objective of the survey in the introduction as this allows the respondent to be informed not only the purpose of the survey but also how the survey results will be used.

When planning a survey there are a number of other factors to consider:

• What do you want to learn from the survey? This normally takes the form of 1 -5 learning objectives
• Who is the target audience for your survey?
• What is the purpose of the final report?
• Who is the end reader?
• What are the potential actions that could be taken based on the results of the survey?

Once the planning stage has been completed then you can move onto survey design.

Keep surveys shortIn terms of survey design, it is best to keep surveys short. A rough guide is that a survey should take less than five minutes to complete. The longer the survey is, the greater the potential of survey fatigue setting in which can lead a respondent to just click any answer to get the survey completed thus providing noisy data or not completing the survey leading to partial responses which then have to be cleaned out of your data. An average respondent can answer five quantitative questions a minute and two open-ended (text questions) a minute.

When designing the questions for your survey it is important to keep in mind the objective of the bigger project but also the objective of the survey and the elements identified during the planning stage. It is recommended that questions are grouped around the learning objectives and that the questions are designed with the audience in mind.

Think of the survey as a conversationWhile designing and laying out questions think of the survey as a conversation between you and the respondent. Citizens’ panel surveys can deal with a range of issues from the complex such as representation in Council and at local government elections through to the basic day to day issues such as changes to local bylaws to allow dogs to walk in a local park. It is important to use language that panel members will understand. It is best practice to avoid the use of jargon or abbreviations; if abbreviations have to be used it is recommended that what the abbreviation stands for is spelt out. The questions themselves should be short, concise statements which are meaningful on their own. The questions can use a mix of positive, negative and natural language however it is important to avoid leading questions as this will result in bad data.

Provided a response for everyoneOne common design error is not providing a response option for all respondents, this can be as simple as providing a Don’t Know or Not Applicable option. If the response options do not provide for everyone the respondent may choose not to complete the survey meaning you miss out on data or they will choose the response which closest fits them which can lead to noisy data and have an impact on the survey results.

When using rating scales such as a Likert scale which is a common rating tool it is important to keep the scales consistent throughout the survey. This not only applies to the scores i.e. a sourcing scale of 1-5 but also the values of each score, if you change the scale part way through the survey it may confuse the respondent, they could answer a question incorrectly or get frustrated and not complete the survey; all of these impact on the data collected and the results of the survey.

 

As discussed earlier it is recommended that questions are grouped together, one way of doing this is by using a grid. It is recommended that a grid question has no more than five rows.

It is a requirement under the Local Government Act 2002 that local governments have an engagement policy on how it will engage with communities on issues, proposals, assets and other activities or matters that are significant or may have significant consequences; a number of Councils have established citizens’ panels as part of their engagement policy. Citizens’ panels are easy to set up and maintain, are cost-effective and the data collected enables decisions and policy to be informed and backed by research.

Happy Surveying!