TIPS: Salesforce Data Deduplication – Part 1

Data duplication can be a hassle for the users and sometimes we would not know it is exist. However, we can use DemandTools to solve this problem. DemandTools is designed for cleaning existing salesforce data. In this blog, we will look on how to clean up the data by using DemandTools. Standard Account object will be used for testing.


In Figure 1, we can easily recognize that there are two duplicate records(highlighted). After the clean up process, both of the duplicate records will merged into one record.


Now let’s start our cleaning process by using DemandTools ‘Single Table Dedupe’ function shown in the figure below.


In order to run the ‘Single Table Dedupe’ function, we need to do some configuration. First we need to create a ‘Scenario’, followed by ‘Matching’ process and ‘Results Grid and Merging’ process. Let’s start by creating a ‘Scenario’.

From Figure 3, there are two methods to select an object.

  1. Firstly, let’s look at the ‘Scenario’ section on the left panel. We called it as pre-build scenario, which provides the readily created method or fields.
  2. Secondly, create our custom scenario from the drop-down list.


The difference between these two methods is, item #1 has limited object to select and incapable of displaying custom object. In this case, we will use #2 to select the object from the drop-down list.


Next, we will proceed to ‘Matching’ configuration. Here, we will need to specify the criteria to find the duplicate records. Okay, what does that mean? In a way, it means that we will need to specify the fields to define the duplicate records in salesforce.


For example, I have 2 records in salesforce Account object, both of them have similar account name. So, I need to specify the ‘Name’ field in this ‘Matching’ section, which helps me identify all the duplicate ‘Name’ data in my salesforce account database. The figure below shows how I configure the ‘Matching’ on a step by step basis.


1. Click on the ‘Add Mapping Condition’ button to add matching fields.
2. Select the search algorithm(Fuzzy, Transpose & AlphaClean). Easy right?


3. Go to ‘Results Grid and Merging’. In this step, demandTools will return two duplicate records shown in the figure below. The fields displayed came from the fields that we selected in the ‘Scenario’ step. Fortunately, DemandTools also provides the capability to add more fields functionality in this step. Therefore, we do not need to go back to the previous step and redo everything just for the sake of adding in more fields. What you can do is, just go to the ‘Grid Setup’ shown in Figure 6, and click on ‘show more field’ button.


4. We need to define which field to merge. First, right click on the ‘Duplicate Key: astrl_ns_tst’ and then select ‘Setup Cutom Merge’ shown in Figure 7.


Yes, you can also double click on the ‘Setup Custom Merge’ field to decide which fields and record to be retained.(Please set the record to Master to keep the record). In Figure 8, the shown fields will be merged and the master record will be retained.


We are almost finish.

5. Now, once we have decided which record and field to keep then click on the ‘Accept’ button on ‘Setup Custom Merge’ window.
6. Lastly, click on the ‘Merge Checked Records’.


Yeah, we are done!


Let’s go back to Salesforce. Now you will only find the record Id ‘001G000000qHdSB’ exist and we have the Phone, Fax, and State/Province merged correctly.