Tracking progress through a survey
[THIS DOCUMENT IS UNDER CONSTRUCTION AND IS BEING CHANGED AND UPDATED]
Overview
Progress through a survey can be tracked per panellist using outcome codes which can be used to show:
- when an email invitation is sent
- when the panellist opens the email
- when the survey is started
- when the survey is complete (including Screen Out and Quota Full)
At the end of the survey period, the survey can be closed by setting the outcome for any panellist who has not opened the email invitation to show that no answer was received.
Throughout the survey period, outcome code counts can be viewed using the Targets/Results window in Sample Manager or as a report in CrossTab.
Setting up outcome codes for tracking
The following outcome codes are required:
- Issued for Sample (system outcome code -1)
- Email Sent (system outcome code -3)
- Email Opened
- Survey Started
- Complete
- Screened Out
- Quota Full
- No Answer
It is recommended that these outcome codes are set up with the following settings:
| Outcome Code | Update Priority | Exclusion rules |
| Issued for Sample | 0 | Check all options. |
| Email Sent | 0 | Check all options. |
| Email Opened | 1 |
‘Allow update with same priority’ and ‘Allow update with lower priority’ should be unchecked (therefore not allowed – this will prevent multiple Email Opened outcomes).
|
| Survey Started | 2 | ‘Allow update with same priority’ and ‘Allow update with lower priority’ should be unchecked (therefore not allowed – this will prevent multiple Survey Started outcomes). |
| Screened Out | 3 | ‘Allow update with lower priority’ unchecked. |
| Quota Full | 3 | ‘Allow update with lower priority’ unchecked. |
| No Answer | 3 | ‘Allow update with lower priority’ unchecked. |
| Complete | 4 | ‘Allow update with lower priority’ unchecked. |
For example, the Email Opened outcome code Update Priority and exclusion rules:
Screened Out, Quota Full and No Reply are all at the same level to allow the panel manager to re-send the survey to panellists with these outcomes, with a different outcome possibly being logged if the panellist chooses to re-start the survey.
The exact settings for the outcome codes will depend on the requirements for your panel.
These rules would also mean that an Email Sent outcome will not be updated for any subsequent mailings if the current outcome code is priority 1 or above. (An Email Sent event will still be logged).
Setting up the sample job and email
When a sample is run and the panellists selected, the outcome codes are automatically set to Issued for Sample (-1).
In order to find out when the panellist has opened their email, the email sent out from Q_Panel needs to use a tracking pixel hidden in an image. This pixel should contain a call to the backfeed URL using the Email Opened outcome code and the response id as parameters.
Note that capturing the Email Opened outcome relies on the panellist opening the email with images allowed - some email suppliers (such as Outlook and yahoo) allow users to block the download of images.
Add the following to the HTML tab of the email:
<img alt="" width="0" height="0" src="https://website base address/MARSC.NET/ClientBackfeed/OutcomeResult.aspx?rid=[EmailResponseId]&occ=7"/>
In this example, the Email Opened outcome code is 7. The response id for each panellist is added using the [EmailResponseId] merge field.
For example:
In order to track when the email invitation is sent an outcome code must be selected to apply after successful send when the mailing is scheduled - we recommend that the system outcome code Email Sent is used.
Setting up the Q_Survey questionnaire
Set up the Q_Survey questionnaire to update outcomes:
- when the panellist clicks on the survey link, update with the Survey Started outcome code
- when the panellist completes the survey, update with the Complete outcome code
- or if the panellist is screened out, update with the Screened Out or Quota Full outcome code
Within Q_Survey there are a number of ways the outcome codes can be updated:
- the outcomes could be updated using scripts to call the backfeed URL or the backfeed API
- when the questionnaire is opened a call should be made to update with the Survey Started outcome
- when the survey is finished, a call should be made to update with the Quota Full, Screen Out or Complete outcome
- the Survey Started outcome could also be updated using the same tracking pixel as above, but with the code for the Survey Started outcome
SCREENSHOTS OF SCRIPT or PIXEL
More information about the backfeed URL can be found here: Introduction to Backfeed URL.
More information about the backfeed API can be found here: http://docs.marscv703.apiary.io/#reference/backfeed
Closing the survey
When the survey period is over, the project can be closed. Any outcome codes still set to Email Sent (or Issued for Sample) can be set to No Answer.
Each outcome code can be defined with a default closure outcome code. For this example we have assumed that all Out For Interview outcome codes use the No Answer outcome as the closure outcome code - you may require different closure outcome codes for each Out For Interview outcome.
More information about closing projects can be found here: Closing a project
The outcome codes from this survey will now be used as part of the response rate variables calculations provided by Q_Panel to provide more accurate statistics on the activity of panel members over various time-periods.
** These views may need some configuration per client???
LINK to Response Rate Variables articles.
Viewing progress throughout the study
Progress can be viewed from the Targets/Results screen as follows:
From the sample job Parameters tab select the Targets/Results button. On the Targets/Results screen click the Show post run data checkbox. Display the outcome codes required by clicking on the Columns button and selecting/deselecting from the report and outcome code columns available.
For example:
The Targets/Results screen will display counts of the outcome codes within each quota cell based on the latest outcome codes.
LINK to Targets/Results article
Progress can also be viewed using a Crosstab report:
Create a new Crosstab report using Outcome Code as the column, dimension/quota cell as the row and a filter which selects the selected sample project (Project Title).
Execute the report:
Select Column% instead of values:
Important
- Capturing the email opened outcome relies on the panellist opening the email with images displayed - some email suppliers (such as Outlook and yahoo) allow users to block the download of images.
- It is recommended that outcome codes are set up with priorities and exclusion rules as described above, in particular, if the email is opened multiple times these settings will prevent later outcomes being overwritten with the Email Opened outcome.
- Subsequent Email Opened outcomes will remain in the backfeed table with an error code (indicating that update was not allowed) - this also applies to any other backfeeds that fail with an error code. This table may need to be cleared out periodically. Alternatively, the backfeed service can be configured to remove these records after processing - this configuration option should only be used once the outcome codes settings have been thoroughly tested to ensure behaviour is as expected.
- Documentation for v7.04 APIs can be found at http://docs.qpanelv704.apiary.io.