This vendor-written tech primer has been edited by Network World to eliminate product promotion, but readers should note it will likely favor the submitter’s approach.
Today, databases are the primary system of record, and organizations are required to keep an accurate picture of all the facts, as they occur. Unfortunately, traditional databases are only temporal and cannot provide a truly accurate picture of your business at different points-in-time.
What organizations need today, particularly in regulated industries, is support for bitemporal data. With a bitemporal database, you can store and query data along two timelines with timestamps for both valid times—when a fact occurred in the real world (“what you knew”), and also system time—when that fact was recorded to the database (“when you knew it”).
Today, more and more organizations need time-based insight—which also helps organizations with security and data governance because they are keeping a better handle on historical data. Only with a bitemporal database can organizations maintain a complete and accurate picture of the past to understand exactly who knew what and did what, when.
A bitemporal database is much more powerful than a temporal database because it allows you to query across system and valid time axes. You can go back in time and explore data, manage historical data across systems, ensure data integrity, and do complex bitemporal analysis with ease. For example, temporal database can answer, “Where did John Thomas live on August 20th?” However, a bitemporal database can answer, “Where did John Thomas live on August 20th as we knew it on September 1st?”
Bitemporal works by ingesting bitemporal documents that are managed as a series of documents with element range indexes for valid and system time axes. Documents are stored in any number of protected collections that are controlled by security permissions. The initial document inserted into the database is kept and never changes, allowing you to track the provenance of information with full governance and immutability.
Here are some questions to help you determine if you should investigate bitemporal database technology:
* Can you afford to lose historical data about your business? In our age of hefty regulatory fines and time-consuming, costly litigation, a database must hold up as the main system of record. Unfortunately, traditional databases do not keep a complete history of the past—they are essentially losing data. Financial companies have been on the leading edge of the use of bitemporal capabilities, and they serve as models for the benefits of bitemporal data management. For example, bitemporal helps large financial companies better manage their data so that they can respond to new regulation and audits, and better manage their risk.
* Are there cases when historical data needs to be updated? Any company that must update historical data would benefit from bitemporal support. For example, in the financial trading business, trades can be amended after they are made. If there is no time-based record of these changes, future analysis of trading positions could be compromised—and organizations could face major fines by regulators. With a unitemporal database, updates overwrite historical data, which can introduce enormous risk to both individual traders and entire companies. Bitemporal design provides an accurate picture of the entire lifecycle of a trade review, including when changes to counter-party names, transaction IDs or price corrections occurred.
* Are you in an industry that requires presentation of historical data to regulators? We live in an age of increasing regulation, with no sign of abatement, and many large organizations are having to go through massive change. The financial services sector, for one, has had to make considerable changes since the financial crisis in 2008, but organizations in all industries are seeing an increased call for historical data. Bitemporal enables organizations to preserve data history, including the changes made to data, so it is easy to collect and present required information for audit and other purposes.
* Are you in an industry in which the sequence of events--and the recording of those events--is critical to security and intelligence, as well as to overall strategy? Many, if not all, industries can benefit from support for bitemporal. In the healthcare industry, for example, bitemporal capabilities can provide a more accurate picture of a patient’s history and help connect the dots between an illness and its antecedents. Likewise, government intelligence and law enforcement agencies can use databases with bitemporal support to make sense of seemingly disparate data, enabling them to better understand motives and even better predict future events. Businesses can tap into this data to determine customer patterns and behavior, for more strategic development of new products and services.
* Is the cost and complexity of storing, accessing and securing historical data increasing? While the cost per gigabyte of data is decreasing, organizations today are spending more on storing historical data because they are dealing with so much more of it. Organizations tend to hoard data in general, but regulations often demand it. Bitemporal data management helps keep storage in check because it avoids the need to set up additional databases for historical data.
* Is the complexity of managing, accessing and securing historical data becoming overwhelming to your database developers? Developers just want to be able to write queries that can easily access historical data. If you aren’t implementing bitemporal and you aren’t tracking important components of historical data, developers won’t be able to ask certain questions of the data. Further, it can be quite complicated to manage different timelines and versions in a traditional database, placing an additional burden on developers and the DBAs tasked with maintaining the data. Bitemporal design helps simplify data access because it’s much less complex.
* Is your company considering the whole picture of data governance? There is an aspect of bitemporal data management that is important to data governance. Bitemporal helps companies improve the security of their data in one key way: If you’re using a traditional approach to managing historical data, you’re going to be making lots of copies of your data warehouse at different points in time and storing that data offline somewhere. The more copies of your data, the more vulnerabilities you’re introducing. With bitemporal, you’re creating a more streamlined, simple approach to maintaining historical data because your data is all in one place. That means you don’t have to keep track of all these copies of data floating around, which helps improve your overall data governance.
In the end, bitemporal data management is about much more than historical data. It’s about strong data governance overall. A strategy must be in place for maintaining every aspect of data governance: security, privacy, lineage, provenance, data quality, compliance, lifecycle management and more. So, it’s important to understand how a relatively simple feature like bitemporal can be so critical to the whole picture of data governance that every organization today needs—and the pitfalls you can run into if it’s ignored.