Build Your AI Empire On Firm Data Architecture Footing

An International Data Corporation (IDC) survey earlier this year revealed that more than 50% of responding business leaders will prioritize the adoption of new AI-powered intelligent automation capabilities over the next several years. These new capabilities, such as generative AI, represent an exciting opportunity for businesses to achieve greater value through improved automation and “self-optimizing” applications. Unfortunately, based on my experience working with architects in recent years, very few have invested time in defining their organizations strategy for data architecture, the consequence of which will result in businesses implementing actions against AI “hallucinations” instead of legitimate learning and insights. 
‍

Key Data Architecture Decisions

After spending the last decade helping dozens of companies architect and implement Pega intelligent automation solutions, I have concluded that data is the #1 culprit that dooms intelligent automation projects. When teams avoid making difficult data architecture decisions at the beginning of these projects, I can guarantee that they will need to refactor big chunks of critical features and functionality of their Pega implementation. All of this additional rework and refactoring could be avoided by making three intelligent decisions early on in architecture and roadmap discussions:

• Define and agree to reporting requirements up front. So often, data architecture is left to fall in place as developers see fit when they are building out business processes. Reporting and storage needs are ignored until late in a project in favor of getting early momentum in the implementation. This often results in data modeling that does not support the availability and integrity needs of your organization. Many architects don’t realize the importance of the reporting requirements in how the data model will be structured during development. While Pega case processing can display any values stored within the processing screens at run time, only fields optimized for reporting or stored in dedicated tables can take full advantage of Pega's powerful reporting tools. These components, which are used to populate dashboards and search functionalities, are often a major deciding factor in the business user’s adoption of your new Pega solution. Improperly defined and executed requirements, however, will negate the value of the platform and hinder organizations from achieving the intended business objectives.  ‍
  

• Deepen your understanding of how Pega stores and processes data.While Pega is a platform built on top of a database and can store any data elements, it is not always the best decision to use that database as your primary data store. By getting familiar with Pega’s real time data referencing capabilities and how compression is used to store case data you can make an informed decision about how your application and its data fits into the bigger picture of your enterprise data architecture. We recently worked with a client who was copying the static details around claims and vehicles into their Pega warranty implementation for case processing. While this did achieve the goal of data availability, it produced unnecessary growth in their storage consumption and created additional complexities for enterprise level reporting. By using Pega’s real time data referencing capabilities to read static and reference data directly from their data warehouse, they could have better managed their data footprint. Writing the Pega-generated data back to that same warehouse would enable improved data availability across the enterprise and simplify the creation of reports.

• Establish an upfront forecast and plan for how data will grow over time. Many businesses don’t plan for data growth in Pega early enough to avoid the pain that unmanaged growth can cause for their Pega cloud infrastructure. Understanding data retention requirements is important, but planning for that retention while managing the growth of the data footprint is often left until licensed space has been consumed. Architecting around a single source of truth using data warehousing can enable you to maintain peak performance on your Pega application while meeting data retention needs in parallel. You will need to ensure that your network infrastructure can handle the transfer of data and design your Pega solutions accordingly.
  ‍

While it is tempting to jump headfirst into application development to build out a new Pega solution, resist the temptation and focus instead on first building your plan for data architecture. Taking this step early is critical to ensuring long-term stability and scalability of Pega implementations as well as ensuring your enterprise is equipped to take advantage of new intelligent automation technologies.