Why Can’t We Use Our Existing Systems Databases for Business Intelligence

In an effort to start framing the many considerations for data souring of a Business Intelligence Solution, first we must look at some of the reasons it is not desirable to simply use existing solution databases.

As a simple starting point, the assumption is that the business intelligence solution that is being sought is envisioned to permit the business users a simple way to analyze, explore and report on information. The information is expected to improve business performance by delivering better decision making throughout your entire organization. Lastly, this improvement assumes that decisions made within the organization will be made as a result of careful consideration of the facts being presented by the information that our business intelligence solution will provide.

With the goals of a business intelligence solution briefly defined, let’s look at the common characteristics of existing systems database designs. These systems are most often storing information within On Line Transaction Processing (OLTP) databases or even flat files generated by business systems. These types of data structures have the following characteristics:

1. Manages complex transactions designed to enforce complex systems business process and rules
2. Key business logic is located within the systems application rather than the database
3. Another software development effort or a third party has created the systems database and as such both its meaning and structure may change over time
4. Often it is not permissible to make changes or add structures to the systems database
5. The design of the systems database may not have a well formed relational design that normalizes data and enforces integrity both at individual attribute levels and at relationship levels.
6. Contains historical data that does not match current business rules
7. Lacks significant meaningful documentation
8. Optimized for fast narrow query performance (e.g. order details for order number 123)
9. Many don’t contain information on historical changes to the information instead only reflecting the current state of the information
10. Usually have been developed in a sandbox with no consideration for consistency and sharing of business logic, entities, and attributes with other systems

 Clearly, with some or all of these characteristics involved, it is important that we find a more suitable source for the information we will present as part of a business intelligence solution. The best option would be to source data from these systems and extract, transform, combine, clean, normalize, denormalized, and otherwise make suitable the data for the business intelligence solutions use. Ultimately, this newly processed data would then be loaded into a data warehouse or data mart. It is the data warehouse or data mart that has been purposefully designed to overcome many of the OLTP data structures limitations for business intelligence usage.