Payment infrastructure for direct debit and card transactions

Description

Before hosted payment gateways existed, I designed and built a bank’s payment engine from scratch: direct debit files sent to partner banks, and card payments settled with a delay.

Key Achievements:

• Built a full payment pipeline from scratch - direct debit and card, years before hosted gateways existed
• Isolated all card data behind a dedicated service - the core system only ever saw the last few digits
• Made file processing crash safe - a crash mid run never lost or duplicated a payment file
• Prevented duplicate debit collections - a customer was never charged twice

Original context

• Two payment types: recurring direct debit files, and card payments settled with a delay
• Every partner bank used its own file format, fixed length or variable length
• Card numbers were isolated in a separate cryptographic service, its own Active Directory identity, its own private keys
• The main system never kept more than the last few digits of a card

Challenges

• Every partner bank’s file format was different, a mistake meant a rejected batch, not a rejected row
• Partner bank FTP servers weren’t always up, a file still had to go out on schedule
• Direct debit instructions could come back rejected or returned, and had to feed back in, not vanish
• Volume spiked hard around certain billing dates
• A crash mid file could not mean a lost or duplicated payment

The solution

Direct debit

• Engine builds each bank’s file format and uploads it
• Rejections and returns feed back into the engine

Cards

• Card data never touches the main system, only the crypto service does
• Processor gets a token, settlement is batched and delayed

Crash safe files

• One transaction spanning the file system and SQL Server
• Process the file, record the results, move the file: all or nothing
• No outbox pattern back then, this was the era’s answer to the same guarantee

No duplicate payments

• Each payment gets a unique reference: account, amount, due date
• Checked against a table of payments already sent
• Already sent, skip it instead of resending

Technology Stack Used

• .NET and C# for the payment engine
• SQL Server for tracking processed payments and files
• MSDTC for the distributed transaction
• FTP for file exchange with partner banks
• Active Directory for isolating the cryptographic service

Conclusions

Every piece here, file formats, card isolation, crash safety, duplicate protection, had to be built, not configured. The tools were different, but the requirement hasn’t changed: money moves exactly once, never zero times, never twice.