Using a dedupe filter in Panel Manager
Dedupe filters allow you to search for duplicate panel members using the fuzzy match filtering system. The results are returned to Panel Manager where the suspected duplicate panellists can be examined using Panel Manager features. The duplicate panel members can then be de-activated, their profile updated or email sent out as appropriate.
Executing a Dedupe filter
From the Panel Manager screen select Filter from the toolbar to show the Filter view. Select the Dedupe Filter radio button.
If there are already Dedupe Filters defined they will be displayed in a dropdown list. Select the filter you require and press Execute. The filter will be executed and the results displayed.
Dedupe filters may also be executed using Filter Manager from the Maintenance menu.
Adding a Dedupe filter
From the Filter view, with the Dedupe Filter radio button selected, click on Add. The Dedupe Filter dialog will appear and a new dedupe filter can be defined.
Enter the properties of the dedupe filter using the following fields:
Filter Name
Enter a brief description of the filter. This is the description that will be displayed in the Dedupe Filter dropdown list, and on the Filter Manager screen.
‘AND’ match fields
By default the conditions will be combined using an ‘AND’ operator, which means that all conditions must be true to return a record. This can be changed to ‘OR’ match fields by clicking on the description and selecting OR. This means that only one of the conditions needs to be true to return a record.
Add Group
Groups may be used as a convenient way of grouping conditions. Although the top-level group can be either set to AND or OR, any further groups added can only be set to AND. We recommend that you spend time considering the filter requirements before starting to define it.
Add Field
To add a dedupe condition click on Add Field within the appropriate Group.
The Profile Variables folder is displayed along with its sub-folders. Variables can be selected by expanding folders to find them, or by using the search function.
The variables that are available for use in dedupe filters are defined in Hierarchy Manager, which can be found within the Maintenance menu. In order to be available for filtering, each variable in the hierarchy must have the ‘Allowed in Named filters’ checkbox ticked.
Rather than specifying criteria based on the selected variable, when creating dedupe filters we need to specify the variables we want to match using the Match Type dropdown list.
Match Type
Match type may be either set to Exact or Fuzzy. Fuzzy matches are only available for alphabetic character-based text fields.
Some text fields may actually contain numbers, such as phone numbers or postcodes. It is recommended that the fuzzy option is not used for this type of data as it may not provide the expected results.
Example – Dedupe Filter
We will create a dedupe filter to find duplicate records where the first name is a fuzzy match and the last name, region and marital status are exact matches.
- Create the filter as shown, entering the filter name, and adding the conditions one at a time by pressing the Add Field button.
- Press OK to save the changes.
You will be returned to the Filter view where the new dedupe filter is selected in the dropdown list ready to be executed.
If the filter is executed and the additional columns Marital Status and Region selected for display, the filter results will be returned and displayed as follows (depending on the contents of your database).
We can see that there are several possible duplicate panel member records. The user may now add more columns to the panel manager display to further investigate and/or email the possible duplicates and use any of the panel manager features as appropriate.
Dedupe filters may also be added using Filter Manager from the Maintenance menu.
Editing a Dedupe filter
To edit an existing dedupe filter from Panel Manager, select Filter from the toolbar to show the Filter view and select the Dedupe Filter radio button. Use the dropdown list to select the required Dedupe filter and press Edit. The Dedupe Filter dialog will be displayed. Expand the top-level group and groups below by clicking on the expand arrows next them, so that the conditions within can be viewed.
- Remove a condition or a group (except the top-level group) by ticking the checkbox next to its name and selecting the Remove button.
- Edit a condition or a group by clicking on its name.
The only change that can be made to a condition is to change the Match Type, which is only applicable to text variables. Groups (apart from the top-level group) can only use the AND operator so no changes can actually be made to groups.
Dedupe filters may also be edited using Filter Manager from the Maintenance menu.