· 20 min read
Series: Event Sourcing & CQRS in Practice · part 4
- 1. Building Scalable and Maintainable Systems with Event Sourcing
- 2. Unleash the Power of Efficiency with Command-Query Responsibility Segregation (CQRS)
- 3. Why Your Event Store Becomes a Bottleneck (And How NestJS Microservices Can Fix It)
- 4. Securing Event-Sourced Financial Systems: From Encryption to Observability
Securing Event-Sourced Financial Systems: From Encryption to Observability
Imagine this scenario: a customer exercises their right to erasure. Delete their data, simple as that. Except the event store is immutable by design. Every transaction, every address change, every login attempt lives in append-only events. You can’t delete anything without breaking the entire audit trail. The DPO gives you two weeks. The regulator issues a seven-figure fine.
This isn’t a specific case I can cite, but the pattern is real — DPAs across Europe have fined companies for inability to satisfy erasure requests, and an immutable event store with raw PII is exactly the kind of architecture that makes compliance impossible after the fact.
This article covers how to avoid that. I’ll walk through encrypting event payloads without killing query performance, locking down CQRS projections so read models don’t leak more data than write models, securing APIs against the attacks that actually hit fintech systems, keeping production data off developer laptops, and building observability that catches breaches before regulators do.
Immutability vs. Privacy
Event sourcing and data privacy regulations pull in different directions. Event sourcing says: “Never delete anything. The history is the truth.
GDPR’s Article 17 gives data subjects the right to request erasure of their personal data (with exceptions for legal obligations and public interest).
PCI DSS requires that cardholder data storage be minimized and protected, with retention limited to business need.
SOX-related SEC rules require retention of certain audit and review-related records for seven years.
The tension isn’t always a direct contradiction — the details depend on your jurisdiction, data types, and what specifically you’re storing. But in practice, if you haven’t designed for these constraints upfront, you’ll find them competing. The rest of this article builds architecture that makes them work together, starting at the event store and working up to the monitoring dashboard.
Layer 1: Securing the Event Store
The PII-in-Events Problem
The most common mistake I see in event-sourced fintech systems is storing PII directly in event payloads. It seems natural. A PaymentInitiated event contains the customer’s name, email, shipping address, and card number. Everything the downstream services need is right there in the event.
The problem surfaces six months later when:
- A customer requests data erasure (GDPR Art. 17)
- An auditor asks who has access to customer names in the event store
- A developer run
SELECT payload FROM stored_events WHERE event_type = 'PaymentInitiated' LIMIT 100and sees raw PII on their screen
The event store was supposed to be an immutable audit trail.
Instead, it’s an unencrypted database of every piece of sensitive data your system has ever processed. And if you’re storing raw PANs in there, you haven’t necessarily violated PCI DSS outright (the standard allows storage of the PAN under specific conditions with proper protection), but you’ve massively expanded your compliance scope and made it nearly impossible to satisfy data minimization requirements in an append-only store.
Pattern: PII Separation with Reference Tokens
The fix is to never store PII in events at all. Store a reference token that points to a separate, access-controlled PII vault. To be clear: no regulation explicitly mandates this exact pattern. But it’s the cleanest architectural approach I know for satisfying the combined constraints of GDPR erasure, PCI data minimization, and event store immutability.
// BAD: PII directly in event payload
const event = {
eventType: 'PaymentInitiated',
payload: {
customerId: 'cust-123',
customerName: 'John Doe', // PII
customerEmail: 'john@example.com', // PII
cardNumber: '4111111111111111', // PAN in immutable store = PCI scope nightmare
amount: 15000,
currency: 'EUR',
},
};
// GOOD: Reference token instead of PII
const event = {
eventType: 'PaymentInitiated',
payload: {
customerId: 'cust-123',
piiRef: 'pii:cust-123:v7', // Reference to PII vault
cardTokenRef: 'tok_4a8b2c', // Tokenized card reference
amount: 15000,
currency: 'EUR',
},
};The PII vault is a separate service with:
- Field-level encryption (each customer’s data encrypted with a unique key)
- Access logging (every read is audited)
- TTL policies (card data expires after settlement)
- Delete capability (GDPR erasure doesn’t touch the event store)
Pattern: Crypto-Shredding for GDPR Erasure
When a customer exercises their right to erasure, you don’t touch the event store. You destroy their encryption key. Without the key, their PII in the vault becomes random bytes — cryptographically erased, mathematically irrecoverable.
interface CryptoShredder {
/**
* Each customer gets a unique data encryption key (DEK).
* The DEK is itself encrypted by a master key in KMS.
* Deleting the DEK makes all data unrecoverable.
*/
generateCustomerKey(customerId: string): Promise<EncryptedDEK>;
encryptPII(customerId: string, data: CustomerPII): Promise<EncryptedBlob>;
decryptPII(customerId: string, blob: EncryptedBlob): Promise<CustomerPII>;
shred(customerId: string): Promise<void>;
}
class KMSCryptoShredder implements CryptoShredder {
constructor(
private readonly kms: KMSClient, // AWS KMS or HashiCorp Vault
private readonly keyStore: KeyStore, // Maps customerId → encrypted DEK
) {}
async shred(customerId: string): Promise<void> {
// 1. Delete the customer's data encryption key
await this.keyStore.deleteKey(customerId);
// 2. Log the shredding event (for audit trail)
await this.auditLog.record({
action: 'CRYPTO_SHRED',
customerId,
timestamp: new Date(),
reason: 'GDPR_ERASURE_REQUEST',
irreversible: true,
});
// 3. The event store is untouched.
// Events still reference piiRef: 'pii:cust-123:v7'
// but decryption is now impossible.
}
}This is the part where I have to be careful about what I’m recommending. Crypto-shredding is a widely used pattern and many organizations treat it as sufficient for GDPR erasure. But the legal picture is genuinely unsettled. The EDPB (European Data Protection Board) has not issued definitive guidance on whether destroying the decryption key qualifies as erasure under Article 17. Some DPAs accept it. Others argue that encrypted data is still personal data as long as the key could theoretically exist somewhere — in backups, key escrow, HSM logs, or disaster recovery copies.
This is not a purely engineering decision. Before you build your compliance strategy around crypto-shredding, you need to:
- Get explicit sign-off from your DPO and legal counsel (not just your CTO)
- Document the full key lifecycle: creation, storage, rotation, destruction
- Prove that key destruction is propagated to all backup systems, replicas, and DR environments
- Verify that no other system retains a copy of the key or the unencrypted data
- Consider whether your specific DPA has published opinions on crypto-shredding
I’ve seen teams treat crypto-shredding as a silver bullet that “solves GDPR.” It’s a strong technical control, but it’s not a legal guarantee. Treat it as one component of your erasure strategy, not the whole thing.
Field-Level Encryption for Event Payloads
For events that must contain semi-sensitive data (transaction amounts, merchant IDs, timestamps that could identify patterns), use field-level encryption within JSONB payloads:
import { createCipheriv, createDecipheriv, randomBytes } from 'crypto';
class EventPayloadEncryptor {
private readonly algorithm = 'aes-256-gcm';
/**
* Encrypt specific fields in an event payload.
* Non-sensitive fields remain queryable in plain text.
*/
encrypt(
payload: Record<string, any>,
sensitiveFields: string[],
encryptionKey: Buffer,
): Record<string, any> {
const encrypted = { ...payload };
for (const field of sensitiveFields) {
if (encrypted[field] !== undefined) {
const iv = randomBytes(16);
const cipher = createCipheriv(this.algorithm, encryptionKey, iv);
const value = JSON.stringify(encrypted[field]);
let ciphertext = cipher.update(value, 'utf8', 'base64');
ciphertext += cipher.final('base64');
const authTag = cipher.getAuthTag();
// Replace plain value with encrypted envelope
encrypted[field] = {
_encrypted: true,
iv: iv.toString('base64'),
data: ciphertext,
tag: authTag.toString('base64'),
};
}
}
return encrypted;
}
}
// Usage in event store append:
const encryptedPayload = encryptor.encrypt(
event.payload,
['merchantName', 'customerRef', 'ipAddress'], // sensitive
customerEncryptionKey,
);
// event.payload.amount remains in plain text (queryable)
// event.payload.merchantName is now an encrypted blobThe result: immutable events with encrypted PII fields, and plain-text business data that projections can still query efficiently.
Layer 2: Locking Down CQRS Projections
The Over-Exposure Problem
In CQRS, projections denormalize event data into read-optimized views. The security risk here is subtle: a projection designed for one use case can accidentally expose data it shouldn’t.
Say you have a MerchantDashboardProjection that aggregates payment events for a merchant portal. It needs transaction amounts and statuses. But if it also includes customer email addresses (because they’re in the original events and “might be useful later”), you’ve just created a read model that gives merchants access to customer PII.
Pattern: Role-Scoped Projections
Build separate projections for different access levels. I mean physically separate projections with different data shapes, not one projection with filtered fields. This isn’t a regulatory requirement per se — neither GDPR nor PCI DSS prescribe your read model architecture. But it’s a practical way to enforce least-privilege access at the data layer, which both frameworks do care about.
// Projection for merchant portal (no customer PII)
class MerchantPaymentProjection {
async project(event: PaymentCaptured): Promise<void> {
await this.merchantView.upsert({
paymentId: event.paymentId,
amount: event.amount,
currency: event.currency,
status: 'CAPTURED',
capturedAt: event.capturedAt,
// NO customer name, email, or card details
});
}
}
// Projection for compliance team (includes audit fields)
class CompliancePaymentProjection {
async project(event: PaymentCaptured): Promise<void> {
await this.complianceView.upsert({
paymentId: event.paymentId,
amount: event.amount,
currency: event.currency,
status: 'CAPTURED',
capturedAt: event.capturedAt,
riskScore: event.metadata.riskScore,
triggeredRules: event.metadata.triggeredRules,
correlationId: event.metadata.correlationId,
// Customer PII resolved at query time via PII vault
customerPiiRef: event.payload.piiRef,
});
}
}
// Projection for operations team (aggregate metrics, no individual data)
class OperationsProjection {
async project(event: PaymentCaptured): Promise<void> {
await this.opsMetrics.increment({
date: event.capturedAt.toISOString().slice(0, 10),
currency: event.currency,
totalAmount: event.amount,
count: 1,
// NO individual payment IDs or customer references
});
}
}Field-Level Authorization on Read Endpoints
Even within a single projection, different users should see different fields. Use NestJS guards and interceptors to strip sensitive fields based on the caller’s role:
@Controller('payments')
export class PaymentReadController {
@Get(':id')
@UseGuards(JwtAuthGuard)
@UseInterceptors(FieldFilterInterceptor)
async getPayment(
@Param('id') paymentId: string,
@CurrentUser() user: AuthenticatedUser,
): Promise<PaymentView> {
const payment = await this.paymentProjection.findById(paymentId);
if (!payment) throw new NotFoundException();
// Ownership check — prevents IDOR
if (payment.merchantId !== user.merchantId && user.role !== 'admin') {
throw new ForbiddenException();
}
return payment;
}
}
// Interceptor strips fields based on role
@Injectable()
class FieldFilterInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const user = context.switchToHttp().getRequest().user;
return next.handle().pipe(
map((data) => {
if (user.role === 'merchant') {
// Merchants see transaction data, not risk scores
delete data.riskScore;
delete data.fraudFlags;
delete data.internalNotes;
delete data.correlationId;
}
if (user.role !== 'compliance') {
// Only compliance sees PII references
delete data.customerPiiRef;
delete data.ipAddress;
}
return data;
}),
);
}
}Projection Rate Limiting
Projections are read-optimized, which also makes them exfiltration-optimized. A compromised API key could dump your entire read model in minutes. Rate limit aggressively:
// Different rate limits per role
@Throttle({ merchant: { limit: 100, ttl: 60 }, admin: { limit: 1000, ttl: 60 } })
@Get('transactions')
async listTransactions(@Query() query: PaginatedQuery) {
// Force pagination — no full table exports
const page = Math.min(query.limit || 20, 100); // Max 100 per request
return this.projection.findPaginated(query.cursor, page);
}Layer 3: API Security for Financial Endpoints
Request Signing for Payment Operations
Authentication alone doesn’t cut it for financial APIs. You also need to prove the request wasn’t tampered with in transit. HMAC-SHA256 request signing is standard practice here (Stripe, Adyen, and most payment processors use it):
import { createHmac, timingSafeEqual } from 'crypto';
class RequestSigner {
/**
* Sign a request payload with HMAC-SHA256.
* Prevents tampering: changing amount, recipient, or any field
* invalidates the signature.
*/
sign(payload: Record<string, any>, timestamp: number, secret: string): string {
const message = `${timestamp}.${JSON.stringify(payload)}`;
return createHmac('sha256', secret).update(message).digest('hex');
}
/**
* Verify incoming request signature.
* Rejects tampered or replayed requests.
*/
verify(
payload: Record<string, any>,
timestamp: number,
signature: string,
secret: string,
): boolean {
// Reject requests older than 5 minutes (replay attack prevention)
const now = Math.floor(Date.now() / 1000);
if (Math.abs(now - timestamp) > 300) {
return false;
}
const expected = this.sign(payload, timestamp, secret);
// Constant-time comparison prevents timing attacks.
// IMPORTANT: use timingSafeEqual on raw buffers, not === on strings.
// String comparison with === can leak length/content info via timing.
const expectedBuf = Buffer.from(expected, 'hex');
const signatureBuf = Buffer.from(signature, 'hex');
if (expectedBuf.length !== signatureBuf.length) {
return false;
}
return timingSafeEqual(expectedBuf, signatureBuf);
}
}Webhook Security
If your system publishes webhooks (settlement notifications, payment status updates), every webhook must be signed. Unsigned webhooks are an open invitation for replay attacks:
@Injectable()
class WebhookDeliveryService {
async deliver(
endpoint: string,
event: DomainEvent,
webhookSecret: string,
): Promise<void> {
const timestamp = Math.floor(Date.now() / 1000);
const body = JSON.stringify({
id: event.id,
type: event.eventType,
data: event.payload,
occurredAt: event.occurredAt,
});
const signature = createHmac('sha256', webhookSecret)
.update(`${timestamp}.${body}`)
.digest('hex');
await this.httpClient.post(endpoint, body, {
headers: {
'Content-Type': 'application/json',
'X-Webhook-Signature': `t=${timestamp},v1=${signature}`,
'X-Webhook-Id': event.id, // For idempotency
'X-Webhook-Timestamp': timestamp.toString(),
},
timeout: 15000,
});
}
}DTO Validation as a Security Boundary
In NestJS, the ValidationPipe is what stands between you and mass assignment attacks. Without strict validation, an attacker can inject fields that shouldn’t be user-settable:
// Command DTO with strict validation
class InitiatePaymentDto {
@IsUUID()
merchantId: string;
@IsNumber({ maxDecimalPlaces: 2 })
@Min(0.01)
@Max(999999.99)
amount: number;
@IsISO4217CurrencyCode()
currency: string;
@IsOptional()
@IsString()
@MaxLength(255)
reference?: string;
// These fields do NOT exist in the DTO.
// If an attacker sends { approved: true, fraudScore: 0 },
// ValidationPipe with whitelist: true silently strips them.
}
// In main.ts — this is non-negotiable for fintech
app.useGlobalPipes(
new ValidationPipe({
whitelist: true, // Strip unknown properties
forbidNonWhitelisted: true, // Throw error if unknown properties present
transform: true, // Auto-transform types
disableErrorMessages: process.env.NODE_ENV === 'production', // Don't leak validation details
}),
);Layer 4: Attack Vectors That Actually Hit Fintech Systems
IDOR on Payment Endpoints
Insecure Direct Object References are the most common vulnerability I see in fintech APIs. The attack is embarrassingly simple: change the payment ID in the URL and see someone else’s transaction.
GET /api/payments/pay_a1b2c3 -> Your payment (200 OK)
GET /api/payments/pay_d4e5f6 -> Someone else's payment (200 OK — vulnerability)
The fix is ownership verification on every endpoint, with no exceptions:
@Get(':id')
async getPayment(
@Param('id') id: string,
@CurrentUser() user: AuthenticatedUser,
) {
const payment = await this.paymentService.findById(id);
// This check must be present on EVERY read endpoint
if (payment.merchantId !== user.merchantId) {
// Log the attempt — this is either a bug or an attack
this.securityLogger.warn('IDOR attempt', {
attemptedResource: id,
userId: user.id,
userMerchantId: user.merchantId,
resourceMerchantId: payment.merchantId,
});
throw new ForbiddenException();
}
return payment;
}Timing Attacks on Fraud Scoring
If your fraud scoring endpoint responds in 10ms for low-risk transactions and 200ms for high-risk ones (because it runs additional checks), an attacker can infer their risk score from response time alone. They adjust their fraud pattern until the response is consistently fast, which means they’ve found a way past your rules.
class ConstantTimeFraudCheck {
private readonly TARGET_RESPONSE_MS = 150;
async evaluate(transaction: Transaction): Promise<FraudDecision> {
const start = Date.now();
// Run all checks regardless of early results
const [ruleResult, mlResult, velocityResult] = await Promise.all([
this.ruleEngine.evaluate(transaction),
this.mlScorer.score(transaction),
this.velocityChecker.check(transaction),
]);
const decision = this.combine(ruleResult, mlResult, velocityResult);
// Pad response to constant time
const elapsed = Date.now() - start;
const remaining = this.TARGET_RESPONSE_MS - elapsed;
if (remaining > 0) {
await new Promise((resolve) => setTimeout(resolve, remaining));
}
return decision;
}
}Event Deserialization Injection
When you replay events from the event store, you’re deserializing JSON that could have been tampered with if an attacker gained write access to the database. Validate event payloads during deserialization:
class SafeEventDeserializer {
private readonly validators = new Map<string, ClassConstructor<any>>();
register(eventType: string, schema: ClassConstructor<any>): void {
this.validators.set(eventType, schema);
}
async deserialize(storedEvent: StoredEvent): Promise<DomainEvent> {
const schema = this.validators.get(storedEvent.eventType);
if (!schema) {
throw new Error(`Unknown event type: ${storedEvent.eventType}`);
}
// Transform JSON to typed object
const event = plainToInstance(schema, storedEvent.payload);
// Validate all fields against decorators
const errors = await validate(event);
if (errors.length > 0) {
this.securityLogger.error('Event validation failed', {
eventId: storedEvent.id,
eventType: storedEvent.eventType,
errors: errors.map((e) => e.toString()),
});
throw new EventIntegrityError(storedEvent.id, errors);
}
return event;
}
}Layer 5: Keeping Production Data Off Developer Laptops
The Anti-Pattern
Developers need to debug production issues. The fastest way is to pg_dump the production database, restore it locally, and start poking around. It’s also the fastest way to fail a PCI DSS audit, violate GDPR, and put an untracked copy of every customer’s financial data on an unencrypted laptop.
This is how data breaches actually happen. Not zero-day exploits. A developer who needed to fix a bug on a Friday afternoon.
Pattern: Masked Staging Databases
Instead of copying production data, create a masked copy where all PII is replaced with realistic but fake data. Foreign keys still work, data distributions look the same, the edge cases are preserved, but no real customer data leaves the production environment. Your developers won’t even notice the difference most of the time. (Strictly speaking, GDPR and PCI DSS don’t require data masking specifically — they require appropriate technical measures to protect personal data and limit access. Masked staging is one of the most practical ways to satisfy those requirements for development environments.)
# Using PostgreSQL Anonymizer extension
# Define masking rules per column
SELECT anon.init();
SECURITY LABEL FOR anon ON COLUMN customers.name
IS 'MASKED WITH FUNCTION anon.fake_first_name() || '' '' || anon.fake_last_name()';
SECURITY LABEL FOR anon ON COLUMN customers.email
IS 'MASKED WITH FUNCTION anon.fake_email()';
SECURITY LABEL FOR anon ON COLUMN customers.phone
IS 'MASKED WITH FUNCTION anon.partial(phone, 2, $$XXX-XXX-$$, 2)';
SECURITY LABEL FOR anon ON COLUMN payment_cards.card_number
IS 'MASKED WITH FUNCTION anon.random_string(16)';
-- Export masked data (developer-safe)
pg_dump --user=masked_exporter mydb > staging_dump.sqlIf PostgreSQL Anonymizer isn’t enough, Tonic.ai, Snaplet, and GreenMask offer deterministic masking (the same input always produces the same fake output, so foreign keys still work) and subset extraction (only export recent data, which keeps staging databases small).
Pattern: Production Debugging Without Production Data
When a developer needs to debug a live issue, they rarely need the actual data. They need to see what the system did. Give them tools to observe without accessing PII:
// Structured logging with PII redaction
class SecureLogger {
log(level: string, message: string, context: Record<string, any>): void {
const sanitized = this.redactPII(context);
this.logger[level]({
message,
...sanitized,
timestamp: new Date().toISOString(),
traceId: this.traceContext.getTraceId(),
});
}
private redactPII(context: Record<string, any>): Record<string, any> {
const redacted = { ...context };
const piiFields = ['email', 'name', 'phone', 'cardNumber', 'ssn', 'address'];
for (const field of piiFields) {
if (redacted[field]) {
redacted[field] = '[REDACTED]';
}
}
// Mask IDs to allow pattern matching without identification
if (redacted.customerId) {
redacted.customerId = this.hash(redacted.customerId).slice(0, 8);
}
return redacted;
}
}Redacted structured logs, distributed tracing via OpenTelemetry, and error tracking (Sentry with PII scrubbing enabled) cover most debugging scenarios. In my experience, developers stop asking for production database access once these tools are set up properly.
Layer 6: Observability as a Security Layer
Metrics That Catch Breaches
Most breaches aren’t caught by firewalls. They show up as weird patterns in metrics: someone querying the event store at 3 AM, a spike in 403s on payment endpoints, a projection rebuild that suddenly takes ten times longer. Set up Prometheus and Grafana to catch these:
# Prometheus alerting rules for fintech security
groups:
- name: security_alerts
rules:
# Unusual data access volume (potential exfiltration)
- alert: HighDatabaseReadRate
expr: rate(pg_stat_user_tables_seq_scan_total{table="stored_events"}[5m]) > 100
for: 5m
labels:
severity: critical
annotations:
summary: "Unusually high sequential scan rate on event store"
description: "{{ $value }} scans/sec on stored_events — potential data exfiltration"
# API enumeration attack
- alert: PaymentEndpointEnumeration
expr: |
rate(http_requests_total{endpoint=~"/api/payments/.*", status="403"}[5m]) > 10
for: 2m
labels:
severity: warning
annotations:
summary: "High 403 rate on payment endpoints — possible IDOR scanning"
# Suspicious off-hours access
# NOTE: hour() returns UTC in Prometheus. Adjust offsets for your timezone,
# or use a recording rule that maps to local business hours.
# Also watch operator precedence — wrap the OR in parentheses.
- alert: OffHoursProductionAccess
expr: |
pg_stat_activity_count{datname="production"} > 0
and ON() (hour() < 6 or hour() > 22)
for: 5m
labels:
severity: warning
annotations:
summary: "Production database connections detected outside business hours (UTC)"
# Projection rebuild anomaly
- alert: ProjectionRebuildAnomaly
expr: projection_rebuild_duration_seconds > 5 * avg_over_time(projection_rebuild_duration_seconds[7d])
labels:
severity: warning
annotations:
summary: "Projection rebuild taking 5x longer than average — possible data integrity issue"Audit Logging for Compliance
Every action that touches sensitive data needs an audit entry. I’m talking about a separate, immutable audit stream that compliance teams can query independently, not your application logs:
@Injectable()
class ComplianceAuditService {
constructor(
private readonly auditStore: AuditEventStore,
) {}
/**
* Log a data access event.
* Called by every service that reads customer data.
* Required for SOX, PCI DSS, and GDPR accountability.
*/
async logDataAccess(access: DataAccessEvent): Promise<void> {
await this.auditStore.append({
timestamp: new Date(),
actor: {
userId: access.userId,
role: access.userRole,
ipAddress: access.ipAddress,
userAgent: access.userAgent,
},
action: access.action, // 'READ', 'EXPORT', 'QUERY'
resource: {
type: access.resourceType, // 'payment', 'customer', 'ledger_entry'
id: access.resourceId,
fields: access.accessedFields, // Which columns/fields were returned
},
context: {
endpoint: access.endpoint,
queryParameters: access.sanitizedQuery,
responseSize: access.responseSize,
traceId: access.traceId,
},
});
}
}
// Usage in a NestJS interceptor - automatic audit logging
@Injectable()
class AuditInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const request = context.switchToHttp().getRequest();
const user = request.user;
const startTime = Date.now();
return next.handle().pipe(
tap((responseData) => {
this.auditService.logDataAccess({
userId: user.id,
userRole: user.role,
ipAddress: request.ip,
userAgent: request.headers['user-agent'],
action: request.method === 'GET' ? 'READ' : 'WRITE',
resourceType: this.extractResourceType(request.path),
resourceId: request.params.id,
accessedFields: Object.keys(responseData || {}),
endpoint: request.path,
sanitizedQuery: this.sanitizeQuery(request.query),
responseSize: JSON.stringify(responseData).length,
traceId: request.headers['x-trace-id'],
});
}),
);
}
}Distributed Tracing Across Event Flows
In event-sourced systems, a single user action triggers a chain of events across multiple services. Without distributed tracing, debugging a security incident means manually correlating timestamps across dozens of log files. I’ve done that. It’s miserable. OpenTelemetry makes this manageable:
import { trace, context, SpanKind } from '@opentelemetry/api';
class TracedEventHandler {
private readonly tracer = trace.getTracer('payment-service');
async handlePaymentCaptured(event: PaymentCaptured): Promise<void> {
// Create span that links to the original command's trace
const span = this.tracer.startSpan('handle_payment_captured', {
kind: SpanKind.CONSUMER,
attributes: {
'event.type': event.eventType,
'event.id': event.id,
'payment.id': event.paymentId,
'event.correlation_id': event.metadata.correlationId,
// Never put PII in trace attributes
},
links: [{
context: this.extractTraceContext(event.metadata),
}],
});
try {
await context.with(trace.setSpan(context.active(), span), async () => {
await this.ledgerService.recordCapture(event);
await this.settlementService.queueForSettlement(event);
await this.notificationService.notifyMerchant(event);
});
span.setStatus({ code: 0 }); // OK
} catch (error) {
span.setStatus({ code: 2, message: error.message }); // ERROR
span.recordException(error);
throw error;
} finally {
span.end();
}
}
}The correlation ID flows from command to events to projections to API responses. When a security incident occurs, you query Jaeger or Grafana Tempo with the trace ID and see the entire chain: who initiated it, what data was touched, which services ran, and when.
The Security Checklist
Before you ship, verify each layer:
| Layer | Check | Tool/Method |
| ---------------- | --------------------------------------------- | ------------------------------------------------------------------- |
| Event Store | No raw PII in event payloads | Grep for email/name/card patterns in JSONB |
| Event Store | Crypto-shredding tested | Trigger erasure request, verify decryption fails |
| Event Store | Field-level encryption | Inspect stored_events table — sensitive fields are encrypted blobs |
| Projections | Role-scoped projections | Different API keys return different field sets |
| Projections | Rate limiting active | Hammer an endpoint — should get 429 after threshold |
| API | Request signing enforced | Send unsigned request — should get 401 |
| API | DTO whitelist validation | Send extra fields — should get 400 or fields stripped |
| API | IDOR protection | Access another merchant's resource — should get 403 |
| Developer Access | No production credentials in dev environments | Audit CI/CD secrets and .env files |
| Developer Access | Masked staging database | Query staging - verify no real PII |
| Monitoring | Anomaly alerts configured | Simulate high read rate — alert should fire |
| Monitoring | Audit trail active | Access a resource — verify audit entry exists |
| Monitoring | Distributed tracing | Follow a trace ID from command to projection update |What I Got Wrong (And What I’d Do Differently)
Looking back at the fintech systems I’ve built, my biggest security mistakes were all variations of the same thing: treating security as a layer to add later instead of a constraint to design around from the start.
The PII-in-events problem? We had to retroactively migrate six months of events to use reference tokens. The projection over-exposure? We discovered it during a penetration test, not during design. The developer access to production? We didn’t fix it until after a near-miss incident.
These patterns aren’t theoretical. They’re the patterns I wish I’d implemented from day one. Retrofitting security into an event-sourced system costs significantly more than building it in upfront, for a simple reason: event stores are immutable, and you can’t unflatten PII out of a million events.
If I were starting a new fintech project tomorrow, the PII separation pattern would go in before the first event gets persisted. Crypto-shredding before the first customer signs up. Role-scoped projections from the first read model. Production access controls before the team grows past five. I learned each of these the hard way. You don’t have to :)