For some people, talking about data integration is about as exciting as watching paint dry (truth be told, some may even prefer watching the paint-drying process!).
Nevertheless, data integration goes on all around us in our increasingly “always-on,” data-connected lives. One familiar example is when you’ve purchased a new mobile phone. You (or an obliging friend / tech-friendly person) probably imported or synced your list of contacts, important dates, ringtones, and other data. What about the mobile apps and music library? Yes - I need that data migrated to my new device immediately, if you please.
Modern nonprofit organizations have their own data integration challenges. In the course of supporting their missions and engaging donors, here are some examples of data exchange scenarios:
Import event registration data for constituents into Blackbaud CRM™ from a third-party event management provider.
Export Mailing Lists with constituent and other data from your CRM system to a third-party mailing services vendor.
Import (or synchronize) Gift / Revenue records from a field office or third-party mail processing vendor.
Data integration includes imports, exports, and other data transfer processes. In this post we’d like to suggest 5 ways to help improve your data integration activities.
Know the Way
When you travel, you use a map so that you don’t get lost. If you don’t want your data to get lost, make sure you know the route it will be traveling. In data integration projects, it is always a good practice to create a “data map” which shows:
Source of the data (which application, location, etc.) which will be transmitted in the process, including:
All data fields
Field types
Field formats and constraints
Destination of the data (again - application, location, etc.) - with details similar to above
Any processes or steps that may occur between the source and destination which will add to, enrich, change, or reformat any of the included data.
Having an analyst create the “data map” document provides the organization, partners, and / or vendors with a clear picture of how your data gets from its source to its destination.
Know the ‘Why’
It is crucial to connect a data integration process to the business purpose for that process. For example, an organization may have a process for importing lists of updated phone numbers into Blackbaud CRM™. The purpose for this process is likely straightforward: to efficiently update the CRM database with the most accurate contact phone numbers.
Suppose that a proposal is made to bring an additional list of updated phone numbers from another data source. Someone may make the assumption, “Great! We already have a process for integrating that data. Let’s use a similar process for the second phone list!” While it is possible that the work done on the first phone import process may help with creating a second similar process, it’s not necessarily true that another similar process will best satisfy the ‘Why’ for the process. Hypothetically, what if the organization found that their CRM system encounters duplicate updated phone numbers between the first and second data import processes? The unintended consequence: adding the second updated phone import process has introduced additional inefficiency to the overall business process.
If you lose sight of the ‘Why,’ it may be more difficult to find the best ‘Way’ (see number 1 above).
No Excess Baggage
Have you ever gone on a trip for an extended period of time that involved frequent moving from place to place? Have you ever thought, “Wow, maybe I shouldn’t have packed my baseball glove, 2 three-piece suits, 8 neckties, and 8 hardback books in my suitcase” (or something like that).
Overpacking for a trip makes it harder when you travel, and having extra data as part of an integration process sometimes adds unnecessary complexity to the process. What does this mean in more concrete terms? If the data import into your CRM system only requires 10 fields from the data source, don’t include 15 or 20 fields of the source data! You may have a reason for including extra fields in an import or export, but you should make sure it’s a very good reason. Transmission and processing of data always comes with a cost in terms of computing resources and time.
When avoidable, don’t overpack. Think of hauling an overfilled suitcase up 5 flights of stairs.
Tidy Up
As the saying goes, “Garbage in, garbage out.” If you don’t want your organization to do extra work, implement data cleansing practices for finding and fixing invalid or inaccurate data.
Practice Data Profiling. Utilize Data Quality tools or develop custom reports on your data sources to show data values that may be inaccurate or invalid. The best time to find bad data is before a process tries to load that data into your systems.
In particular, ensure data values are valid and properly formatted. This is especially important for email, date, and monetary fields. Many database applications, including Blackbaud CRM™, will not allow a record to be imported with an incomplete date or an invalid email address.
Having quality checks and cleansing before importing data into your system will result in significant savings of time and money.
Verify
Despite the best efforts of everyone, it is quite likely that some inaccurate data will get into your database. What can you do about it? Similar to item 4 above, there must be ways to:
Identify inaccurate data. Utilize Data Quality tools or develop custom reports to find bad data.
If possible, identify which processes (import, integration, or other) introduced the inaccurate data.
Develop plans for improving the processes and minimizing the occurrence of bad data in your CRM system.
In summary, data integration is a critical part of your organization’s business processes. Implementing the above data integration practices can help accomplish the following:
Minimize tedious, slow (and costly) effort.
Maximize efficiency, accuracy, and reliability.
If you’d like to know more about how BrightVine Consulting can assist your organization with your data integration projects, please contact us.