Sample Manager Parameters

The Sample Manager Parameter tab contains the key project parameters and the dimensions required for the sample.

Project parameters

Some key project parameters are addressed first.

Title

This is a mandatory field. The title of the sample must be unique within the prototype.

External Reference

This is a text box to record an associated project or cost code for administrative purposes. You can enter any text up to 20 characters in length. This field can be used to cross-reference the sample project with data held in other systems. It is not mandatory.

Project Date

This is the planned date for interviewing. This is mandatory and by default it is the create date.

When research is conducted over a period of more than one day enter the mid-point of the research period.

Report Only

To run a feasibility check, Sample Manager provides a Sample Report of available sample without actually issuing that sample. This allows you to check sample availability and then if necessary to refine your sample project specification. Panellists in your database remain available to other sample projects.

By default the sample project is Report Only. Make sure that this is checked before you submit it unless you want QuenchTec Panel to actually issue sample and create a sample output file.

Targets and Rates Calculations

Target Interviews

This is the number of usable completed interviews or questionnaires that the research requires. The default value of zero must be changed to a non-zero value or the sample project cannot be submitted to the Job Queue.

Sample Manager combines a sophisticated response rate analysis with a user-supplied incidence rate to predict the actual sample volume that needs to be contacted in order to meet the target interview requirement.

To specify a sample volume directly put it in the target interview field and leave the response and incidence rates at 100% (default).

Incidence Rate %

This is a single rate applied across all quota cells. It is the rate of usable complete interviews (Target Interviews) that you expect from those people who have agreed to be interviewed.

The Incidence Rate % is intended for use when screening questions at the start of the interview exclude panellists who have agreed to the interview, being screened out or quota full.

Response Rate %

This is a prediction of how many panellists need to be contacted in order that the right number agree to be interviewed.

Accurate response rates help to draw the right sample to complete your sample project without either wasting sample or running out and having to come back for more.

Sample Manager has two options for Response Rate %:

Fixed

You can supply Sample Manager with a response rate. For example, if your experience tells you that you need to contact five people for each one who responds select Fixed and enter 20%.

Per Cell

Alternatively Sample Manager can calculate a response rate for each quota cell where there is enough history available in the panel database via outcome codes.

It makes better use of the available sample by taking into account the historic propensity to respond of your panellists. This method avoids over-specifying sample for the categories of panellist that are more likely to respond and it boosts the sample issued for those that are harder to reach.

The calculation is specific to a database and will typically be defined when the database is built.

The response rate calculation for a database is defined as part of implementing QuenchTec Panel. The response rate numerator and denominator are defined based on outcome codes. The denominator is the base number of invitees and the numerator is a count of panellists historically willing to be interviewed.

For the calculation to be meaningful, a minimum of 50 interview outcomes must be recorded in the database for that cell. When fewer than 50 records are present for a cell, the figure that you enter in Response Rate % is used for that cell.

For example: you select Per Cell and enter a Response Rate % of 25%. For the particular cell married/male Sample Manager checks that there is sufficient historic data (more than 50 records) and calculates from all the previous invitation outcomes for married males a Response Rate % of 19.3%. Cells where Sample Manager finds fewer than 50 records will use the 25% figure you specified.

Calculated Target

This displays the sum of the quotas for all the target matrix cells. Normally this will be identical to the value entered above it for Target Interviews.

It shows a value of zero if Sample Manager cannot perform its calculation because a dimension is incomplete (i.e. its Target % is not 100).

If some cell quotas are very small there will be rounding differences so you will see the overall calculated target value slightly above the target interviews value. Sample Manager insists on providing whole people! - and rounds upwards to a whole number, and will meet or exceed the target interviews needed.

The target matrix

A sample that specifies the overall target number of interviews with no dimensions defined would be a simple random sample within the constraints of the universe of panellists in your database.

To apply quota to the sample you must define dimensions and elements. From these Sample Manager constructs a target matrix. Each cell of the matrix represents an interlocked set of dimensional elements.

Number of cells

For example: a sample might be defined with the three dimensions:

• Gender with two elements
• Age with five elements
• Region with four elements.

This will result in a target matrix with:

• 2 + 5 + 4 = 11 elements
• 2 x 5 x 4 = 40 cells

The number of cells increases dramatically as you specify more elements so avoid over-specifying your sample and creating cells which are too specific in relation to the overall size of the sample.

Sample Manager will allow you to specify up to 900 elements, subject to your license. If you need to create target matrices larger than 1,000 cells please talk to the QuenchTec support team.

Type

This determines the nature of the target matrix and the method by which cell quotas are defined from the dimensional targets.  Two options are available – Interlocking and non-Interlocking.

The type selected can be changed later if required.

Interlocking (default sometimes called Nested)

Multi-dimensional quota samples can be onerous to construct manually.  Sample Manager automates this, allowing you to specify your targets as one or more interlocking matrices (e.g. age by sex by region).

If there are shortfalls against cell quotas then these can be observed after submitting the project to the job queue which checks the availability of sample in the database and creates the detailed sample report.

Non-interlocking (sometimes called Non Nested)

Several difficulties with complex interlocked models, such as unachieved targets or empty cells or even knowing what legitimate targets to set (e.g. Males, Divorced, Aged 25-34, Educated to degree level, in the North), can be overcome by using a non-interlocking model.

When processing the job Sample Manager calculates cell quotas using the best of two algorithms (determined automatically).  If there is plenty of sample available in the database then cell quotas are optimized to make best use of the panellists, reducing the quota where panellists are scarce.  If there is insufficient sample Sample Manager provides a ‘best fit’ with shortfalls against the dimension element targets highlighted in the sample report.

Additional client-specific types can be implemented. Contact the QuenchTec support team for more information.

Combining Interlocking and Non-interlocking

Sample jobs may also be defined with a combination of interlocking and non-Interlocking dimensions by using combined dimensions.  This will be explained in more detail in the article about combined dimensions.

Overlap

It is possible for the same panellist to be selected for more than one quota cell.  For instance we might specify a newspaper readership where 50% read the Times and 50% the Guardian but the database allows panellists to be recorded as reading both.  If Sample Manager selects the same individual for two or more quota cells then he/she is overlapped.

This anomaly can only be identified when Sample Manager actually allocates sample i.e. when you submit a job to the Job Queue.  It depends on the specific panellists who are (randomly) allocated.

The overlap option determines the policy to be applied when overlap occurs. By default Sample Manager does not permit overlap.

Sometimes a mistake when entering dimensional ranges leads to unintended overlap. This happens when ranges are defined e.g. as age 20-24, 24-27 etc. The age 24 appears in both ranges. By not permitting overlap Sample Manager can help you identify this mistake.

The Overlap dropdown list on the Parameters tab gives four options:

The Excluded option is the fastest of the options.

Job processing for the options Ordered and Random will be slower if a large number of panellists are overlapped.

Fit to universe

If you maintain a panel database with balanced demographics or have fed your sampling database with the totality of panellists, you may view your database as representative of the population universe for your research.

With the Fit to universe option Sample Manager calculates quotas based on the underlying counts in the database.

This option works with both the Interlocking and Non-interlocking methods.