Introducing the “Smart Field Clear Results Global Change” for Blackbaud CRM™:
Why you need it and when you should use it
Smart fields are a highly used tool in Blackbaud CRM™ assisting in queries, reports, and segmentation. Smart fields are designed to efficiently perform complex calculations and value group assignments (for some or all records), and store the values to be used by the application later. Using smart field values is much more efficient and requires fewer system resources than calculating values on the fly.
To use smart fields accurately, it is important to understand how they work and how they are updated. One thing they all have in common is they are run at a point in time, values are stored, and their “last run on date” is updated. The next time it runs, the smart field will look for incremental changes (records added or updated since last run) only. Generally, this is a good thing as it allows the smart field refresh processes to be more efficient. But as with most things, there are tradeoffs, and at times certain changes can fall through the cracks of the incremental change logic. This post will take you through some of those potential scenarios, and provides a handy solution for periodically executing a full re-run which we recommend.
So what exactly is an incremental update? Or more importantly, what isn’t? This is the million dollar question, and (cue the drum roll), “it depends”. When developing a smart field definition, the developer tries to account for all the variables in their code. But this is a huge task and at times can almost be impossible to envision all the ways a record can be added, edited, adjusted, or deleted. Often the most common scenarios are addressed, but there can be occasions where the underlying data in all tables the smartfield is dependent on does not get updated. It all depends on the smartfield logic which can be a blackbox to the user. For example, with constituent merges or revenue adjustments there can be changes to data that don’t trigger an update to the underlying data’s “date changed” value, and so that change may not be pulled in and things get out of sync. Similarly, when records are updated directly by a script, the script may not trigger a change to the “date changed”, and hence would not be identified as an incremental change.
Finally, there are also times that you may need to update the parameters of the smart field itself, such as when there is a change to your qualifying criteria or logic. For these occasions, organizations often assume they can use the out of the box action to periodically “Clear Results” of a smart field, the expectation often being that every smart field value will be nullified allowing for a clean, full re-run to follow.
However, and this is important, the action of “Clear Results” does not actually remove the stored data from the table. It does reset the smart field’s last run on date to blank/null, and the next time the smart field is processed it will be treated as a full refresh (as opposed to an incremental refresh). But because it doesn’t also do a full dump of the current smart field tables before doing so, there can be misleading and incorrect “orphaned” smart field data that remains in the tables and on records. In practice this can look like a record successfully updating from value A to value B in the re-run, or having value A added for the first time. But where they previously had A and need that value totally removed, the orphan value of A may stick around rather than being nullified.
The other downside of the “Clear Results'' action is that it has to be performed manually smart field by smart field. For these reasons, BrightVine has developed the “Smart Field Clear Results Global Change” which can be configured to reset the last run on date and/or remove the stored values for one or multiple smart fields. This global change is intended for use in a queue as the first step, to be followed by your standard smart field refresh steps. And like any business process, the global change can be manually run or scheduled to run on a recurring basis.
So how often should you use the Clear Results Global Change? The answer is, again, “it depends”. There is not a one size fits all solution. Review your organization’s smart fields and prioritize their business use. The priority is organization specific. For smart fields that are high priority impacting business decisions, accuracy is essential. A code review of these smart fields is recommended to understand exactly what scenarios are being addressed in the incremental update. It might be determined that high priority smart fields should have their results cleared and a full refresh is required with every update. Of course, a full refresh takes more system resources and will need to fit into the organization’s production schedule. On the other hand, with a low priority smart field, performing incremental recurring updates and a full refresh monthly might be acceptable. It is recommended that every smart field have the results cleared and a full refresh performed from time to time to avoid any possible data anomalies.
For more information or to download this helpful global change, please visit our Resources page here.