Data Is as Data Does
“Data is a precious thing and will last longer than the systems themselves.”
Sir Tim Berners-Lee, inventor, world-wide-web
It’s always nice to be able to share a joke with your customer so the other day when I was asked by IT infrastructure staff at a customer about any ‘inconveniences’ our solution would ‘force upon them’.
This is my slightly tongue in cheek reply:
Customer: So where do you manage your authentication information?
I don’t know, where do you manage yours?
Customer: OK, so where do you store your FX rate data?
I don’t know, where do you store yours?
Customer: What about your interest rates?
I don’t know, where do you store yours?
Customer: Reference aliases?
I don’t know, where do you get yours?
Customer: OK, OK – what about managing documentary materials around exception management?
I don’t know, where do you keep yours?
When it got to Access Entitlements I sensed things were about to get heated. It was time to come clean – our policy with Clareti Transaction Control (CTC) is that we always integrate to a customer service when available and recommend that ownership of data within the system is only taken when no suitable service exists; so ‘our data’ is wherever you want us to get it from. We have default built-in implementations of all these services, but we suggest that you use your own services whenever possible.
We have found that this yields enormous benefits for customers, project staff and the product itself
on many fronts:
- The time to deliver projects is naturally reduced as the number of components being introduced to the customer environment is lower; instead known and trusted customer services are being reused. Naturally the time for testing is therefore also reduced, which although we can provide automated deployment tests to our customers is nevertheless a significant component of any project. The test effort is lowered as fewer test cases are required for the slimmer new system and the required tests for the external integration are of a kind that can be efficiently automated.
- Another cause of increased project time around duplicated services is that around defining and testing the manual workflows for data administration and populating such an internal service with an initial data set. CTC allows you to integrate to your BPM of choice. Nevertheless agreeing, defining and testing workflows is always a sizeable task. All of this is avoided when reusing a well-established service and using an existing service can of course save significant costs in terms of the actual day-to-day data administration.
- Project compliance is also typically a major factor, complicating integration and acceptance. There are naturally extensive standards, both internal and industry-wide, which apply to the management of sensitive data. Given the variety of these standards geographically and by sector it is atypical for a vendor’s built-in service to tick all of the boxes required for compliance, which causes project costs associated with negotiating compliance and customising around the product; whereas reusing a service slices away all of these issues.
- Of course using an external service means that the service is already defined and resourced. When duplicating with an internal service there are project costs in terms of configuration of that service, but more significantly that service impacts the overall operational running costs of the system (picking up costs not strictly associated with the project itself in terms of additional hardware for running a highly available and disaster recoverable service and storing the data in question). The latter is frequently a large hidden cost involved in these duplicated services – the volume of data is typically significant and storage in a clustered enterprise environment often requires expensive software and hardware (we know, we help people manage that issue a lot Gresham Storage). This is especially undesirable with a product like CTC which itself, if desired, supports operation without use of a relational database and associated costs.
- From my point of view in product development terms we can concentrate our expertise working where we can add value – in data learning, matching capabilities, data modelling, high performance and resilience and a super intuitive user interface – rather than reinventing wheels with a lowest common denominator feature set. Also writing our own implementations to the same well-defined service APIs keeps us architecturally clean and allows us to keep pushing the boundaries in terms of our automated acceptance, deployment, integration and unit tests. This means the system comes first and you get what you want from us.
I guess our conclusion, from many years of experience in IT systems integration and reconciliations, is that we need to be absolutely focused on making the situation better and never introducing additional opportunities for data errors and mismatches. We are in business to solve customer operational data quality and data flow issues for improved control and lower operational risk. We don’t want to compromise this by meddling with actual data and processes, which should always be kept as unified as possible and located where the core expertise resides. Data Is as Data Does.
Check out – www.ocdqblog.com from Jim Harris who inspired the title and blogs on all things data (@ocdqblog if you are a Twit).
