Rolodex Data Product Architecture
This document outlines the architecture of the Rolodex Data Product, including key components, data flows, and interactions that enable seamless contact management and integration.
Overview
The Rolodex Data Product provides a unified, event-driven approach to managing contact data. It offers RESTful APIs for interaction and publishes events for asynchronous replication to external systems.
Architectural Components
API Layer
- Built in Python with FastAPI, providing a RESTful interface.
- Handles authentication and authorization (JWT token-based).
- Provides endpoints for creating, retrieving, and listing contacts and related entities.
Datastore (Database)
- Uses AWS mySQL.
- Provides persistence for contacts, parties, addresses, relationships, and legacy identifiers.
Event-Driven Integration (Outbox Pattern)
- Uses the Outbox pattern to reliably publish domain events.
- Events (e.g.,
PersonCreated,OrganisationCreated) are stored in an outbox table and asynchronously published.
ElastiCache Redis Event Stream
- Publishes events to Redis Streams.
- Enables external adapters to consume and integrate these events.
Adapters (Consumers)
Adapters consume events from the Redis streams and replicate data to external systems, including:
- Salesforce Adapter: Synchronises contacts using Salesforce's REST API.
- D365 Adapter: Integrates contacts with PowerApps and Dataverse APIs.
Data Flow
Event Flow
REST API Call (e.g. Create Person)
↓
DB persistence
↓
Outbox table
↓
Background Worker publishes to Redis Stream (asynchronously)
↓
Adapters consume events from Redis Stream
↓
External Systems (Salesforce, Dynamics)
Deployment
- Hosted on AWS (ECS Fargate, RDS, ElastiCache).
Developer Experience
- Documentation provided via an MkDocs-based Developer Portal.
- Needs extending with How-tos and various useful guidance, the aim is to make external teams self-serve.
Security & Authentication
- JWT token based authentication implemented for API security.