Wondering how to balance between cost optimization and reliable software in the specific case of FinTech integrations? In this article, we’ll perform the analysis of some software integration testing examples, from a perspective of a developer, and try to draw some conclusions for you.
If you ask me, finance is all about trust. Consequently, success in FinTech also largely depends on it. From a perspective of a software developer, that trust begins with our trust in the code itself. We need to be confident that our code is doing what was required to do. There could be poorly defined requirements, misunderstood compliance items, and so on, but it’s always great to know that we did our part properly and by all standards.
Unfortunately (or luckily), it’s not all about us, since finance is all about cooperation of different entities – institutions or enterprises. That means a lot of system integrations. If two sides are integrating over an API, for example, both will want to make sure the communication is going smoothly and that all required data is being sent. As much as we like automation for its time-saving capabilities, not all parts of this integration testing can be automated. We still need a person to sign off the testing results and confirm that the code on both sides is good for production. In large systems, with a large number of integrations, this can get costly.
Let me share with you my experience on this.
Case Study – Upgrade to 3D Secure version 2
How did we support 3DS authentication in transactions?
During the implementation of the changes required to upgrade to 3D Secure version 2, my team was tasked with (a simple) task of providing the acquiring processors, which our payment processing system is integrated with, with some additional data. You can read more about all changes related to this upgrade and the reasoning behind them here.
One of the changes required by MasterCard was the addition of a global identifier for a 3DS transaction acquired from the directory server – the directory server transaction ID, in the messages sent to the acquiring processor.
Supporting 3DS authentication in transactions is not an easy task in terms of required infrastructure and resources. That’s the reason why a growing number of processors are relying on the services of third-party providers. This was our case, as well. Our provider would do all the hard work of checking the cardholder’s data, authenticating the cardholder and returning us the standardized result, which we can then forward to the acquirers as proof of identity. Handing over 3DS to a third-party really makes sense from a resource management perspective, but implies some limitations when it comes to testing the integrations between processors that don’t share a common 3DS provider.
How we tested it and why it was okay not to have a test directory server…
The Directory Server is one of the crucial components of a 3DS system, providing a centralized database of accounts’ risk scores, transaction history profiles, etc. MasterCard stated they wouldn’t be supporting a global test directory server, so it was down to the integrating parties to decide whether they want to check the logical integrity and validity of this field. This is completely understandable. Even with the test traffic being many times smaller than the one in production, it would still require MasterCard to invest some serious resources into such test environment.
On the other hand, our system has 15 different acquirer integrations and, in the worst case, that would mean 15 different 3DS providers. Creating a test environment, which would integrate all of them, simply wouldn’t be viable. Because of that, the validity of the directory server transaction ID wasn’t checked. Implementing the integration of test directory servers of different 3DS providers would simply be an overkill and a waste of resources to test such a small implementation change. Therefore, the test consisted only of a structural integrity test, with no real validation of field contents against a centralized instance.
… and why it wasn’t okay to have so few test cases.
The problem with this case of integration testing wasn’t the lack of a proper test environment, but the fact that the processor required us to send only a couple of test transactions with rather arbitrary test data. We were free to decide which test cases would be enough to cover the new implementation. We did so, focusing on the new MasterCard fields.
However, after a few months, the processor came back to us and stated that we have a problem with some Visa 3DS 2 transactions and that we’d need to make some changes in the implementation. This wasn’t a huge issue, since it didn’t cause any non-compliance penalties or a large amount of transaction declines, but it could have, so we treated it as a bug, anyway.
3 important lessons we’ve learned so far
Every developer knows trade-offs are a reality. We can’t expect endless resources in developer test environments.
- With manual and automated testing alike, the crucial part is identifying the test cases, which need to be covered in each of these testing phases, while also considering the cost of implementing and maintaining a test environment, which would support them. It makes no sense to invest a huge amount of time in developing mock services and simulators, which would replace third-party services we use in production. This should be a constant discussion between developers, QA and DevOps.
On the same note, we can’t be impatient and neglect creating test cases that would cover the essential parts of the implementation. That means:
- Taking time (and effort) to identify the crucial parameters, which will affect the behavior of the tested component. These are all well-known QA practices, but in some cases, it seems they are ignored when it comes to testing so-called “small changes”, as was the example of sending the additional 3DS fields.
- No matter how small the code change might be, it can have a large effect on how to large integrated systems work together. In my opinion, the best way to avoid this is by identifying a test transaction set, which runs every time a change in the integration is implemented.
Case study – Complete Development in a Month/Wait Certification for the Next 6 Months.
What did we work on?
In this example, we’ll talk about Payment Facilitators. In case you want to know more about them, you can check out this blog post.
The fact that a payment facilitator solicits the payment means that cardholders can have a problem identifying the transaction later on. On their credit card statements they would see a payment to a company they have never heard of and never bought anything from. For that reason, card schemes require payment facilitators to provide the business names and addresses of their sub-merchants, so that cardholders can recognize the transactions and the payment network can avoid unnecessary charge-back claims.
My team worked on implementing the logic, which would send this data to acquiring processors, which would in turn pass it on to the payment network. The implementation itself was rather simple, but the testing and certification process turned out to be the biggest problem.
Comprehensive test data, but no communication.
The test consisted of sending transactions between the two test environments (ours and payment processor’s), taking the transaction identifications returned by their system and putting them into a form provided. Once we would send them the filled out test form together with the files containing transaction settlement data, they would reply only after a month. Because of this, the release was delayed by more than 3 months. I don’t know what was happening on their side. I suspect they had a cumbersome manual validation process and, at the same time, reduced the amount of people doing it to an absolute minimum. Anyway, what matters most is that these very long response times prolonged our non-compliance period. That meant some of our clients had been on hold, which, in turn, meant lost revenue.
From a developer’s point of view, waiting for a month to receive feedback is very frustrating, partly because it’s something that’s out of our control. We can’t say we’ve completed the work, and we have nothing to work on at the same time. Another consequence was that we simply forgot what we’ve changed in the code exactly, so when we’d receive a feedback, we’d have to check our documentation and take some time to get back into the code that stood unreleased, while we worked on other things.
4 worthy lessons we’ve learned
Let’s sum up the key things we’ve learned throughout this experience:
- During certification and integration testing, manual validation is ineffective, but doesn’t require a big upfront investment as an automated solution. The key is in the balance.
- Automated testing is something that can cover the very basics of the integration logic and then expand incrementally, as available resources permit. As mentioned in the conclusion to the first case, identification of a basic set of test transactions, which would cover the most frequent use scenarios, would be a basis for implementing such automated testing, as well.
In the end, we need a person to vouch that something will work as expected, but we need to provide them with the automated tools to do it in a reasonable amount of time.
This brings me to my final conclusion.
- Cost optimization shouldn’t be done at the expense of those you’re working with. Putting partners at the lowest priority isn’t nice from a relationship perspective, and sometimes can be harmful from a business perspective, as well.
Remember, we’re all working towards the same goal. Increasing response times for resolving integration and certification issues is a calculated risk. Still, there’s a limit on how much you should do it - and that limit should prevent the loss of business opportunities.