In the latest instalment of our Payments IQ series, our Head of Business Intelligence Charlot Agius offers a sneak peek at the changes we’re making to take data-driven opportunities to the next level for our customers.
You might think that making a payment is as easy as virtually ‘sending’ money from one place to another. In reality, there are many players and steps involved in every financial transaction. There’s a card issuer (like Ixaris, which self-issues its own virtual cards), there are the card schemes like Visa and Mastercard, there’s a card issuing processor, a card programme manager, and of course the end-user. That’s not to mention the different processes involved from authorising to settling, funding and reporting. The data flowing between all of these players and processes is, logically, connected because it ultimately relates to the same financial transaction. However, that doesn’t mean it’s actually connected. Physically, data sets are often disjointed.
Ixaris’ data team designed a data warehouse to encapsulate such disjointed data sets, creating a single, consolidated view of all transaction data. This data warehouse is a source for multi-dimensional data cubes, which our customers can use to guide their management decisions.
The first data warehouse schema designed for Ixaris had a multi-dimensional and multi-granular schema design, enabling our team to represent a transaction at its highest granular view, with the ability to slice and dice data at different levels. However, when we first designed this data warehouse our most challenging data requirement was near-real-time processing of data into the data warehouse — at a time when only batch processing was associated with data processing in a data warehouse environment.
Traditionally, batch processing is performed on the previous day’s data overnight and takes many hours to complete. We, however, adopted an incremental approach to data transformation, through which each execution performs the processing on the changes or deltas in the source system data. With this method, we could have near real-time data in the data warehouse, with an average five-minute delay from the data source.
We also split our data pipeline into modules, so we could choose to execute modules using either an incremental or traditional approach. At the time, this was an ideal solution to a very real challenge. But today, developments in data warehousing technology, our own 360-degree view of customer needs, and our team’s extensive experience have combined to fuel a whole new, next-generation payments data warehouse platform.
Today, we are re-thinking how we architect and design data and business intelligence at Ixaris to build a next-generation payments data warehouse. This means consolidating our architecture that was previously de-coupled over time. For example, we are exploring a galaxy-schema data warehouse design, that is derived as a collection of star schemas interlinked and completely normalized to avoid redundancy and data inaccuracy. We are also analysing how microservice architecture (in conjunction with docker containers) can be applied to our data pipeline design.
This new design and architecture set Ixaris up to scale easily and cost-efficiently. The technology that we are choosing supports distributed data processing, making it easy to deploy new nodes. With this new architecture, we will also reduce the time-to-market significantly so our customers will benefit from quicker releases of features and improvements.