A scalable, event-driven order processing and inventory management system built to demonstrate cloud-native distributed systems patterns.
This project simulates a high-throughput e-commerce backend (similar to Amazon's Order Execution) using Python (FastAPI) and AWS Primitives (SQS, DynamoDB). It specifically addresses real-world engineering challenges like Idempotency, Eventual Consistency, and Optimistic Locking.
The system follows the Database-per-Service pattern, decoupling services via an asynchronous Event Bus (SQS).
graph LR
Client([Client / Frontend]) -->|POST /orders| OrderService[Order Service]
OrderService -->|Writes| OrdersDB[(DynamoDB: Orders)]
OrderService -->|Events| OrderQueue>AWS SQS: OrderQueue]
OrderQueue -->|Consumes| InventoryService[Inventory Service]
InventoryService -->|Conditional Update| InventoryDB[(DynamoDB: Inventory)]
InventoryService -->|Events| PaymentQueue>AWS SQS: PaymentQueue]
PaymentQueue -->|Consumes| PaymentService[Payment Service]
PaymentService -->|Updates Status| OrdersDB
- Order Service (
/services/order-service): Public API. Hanldes request validation, deduplication (Idempotency), and initial persistence. - Inventory Service (
/services/inventory-service): Background Worker. Reserves stock using Optimistic Locking to prevent overselling. - Payment Service (
/services/payment-service): Background Worker. Processes payments and handles retries. - Infrastructure: AWS SQS (Message Broker), DynamoDB (NoSQL Store), CloudWatch (Logs/Metrics).
- Problem: Network failures can cause clients to send the same
POST /ordersrequest multiple times. - Solution: The Order Service checks the
order_id(orIdempotency-Keyheader) before processing. Duplicate requests are detected and the previous response is returned without side effects.
- Problem: High concurrency (e.g., Prime Day) can lead to race conditions where two users buy the last item.
- Solution: DynamoDB Conditional Writes (
stock >= requested_qty) ensure atomic decrement. If the condition fails, the order is rejected safely.
- Payment Failures: The Payment Service includes a simulated failure rate (10%) to demonstrate fallback logic and Dead Letter Queue (DLQ) usage.
- Out of Stock: Specific SKUs (e.g.,
B456) are initialized with 0 stock to test the "Sold Out" compensation workflow.
| Component | Technology | Reasoning |
|---|---|---|
| Language | Python 3.9+ | Standard for backend services / Lambda. |
| API Framework | FastAPI | High-performance (Starlette), Async I/O, Auto-Swagger docs. |
| Validation | Pydantic | Strong type checking and data validation. |
| Cloud | AWS (Boto3) | SQS for decoupling, DynamoDB for key-value scale. |
| Simulation | Moto / LocalStack | Allows running the full cloud stack locally for free. |
| DevOps | Docker, GitHub Actions | Containerization and CI pipelines. |
You can run this full cloud stack on a single laptop in Simulation Mode (No AWS Creds required).
git clone https://github.com/ShachiMistry/scalable-order-processing-system.git
cd scalable-order-processing-system
make setupOpen 3 terminal tabs to see the distributed system in action:
Terminal 1 (Order API)
make run-order
# Server running at http://127.0.0.1:8000Terminal 2 (Inventory Worker)
make run-inventory
# Listening on OrderQueue...Terminal 3 (Payment Worker)
make run-payment
# Listening on PaymentQueue...curl -X POST "http://127.0.0.1:8000/orders" \
-H "Content-Type: application/json" \
-d '{"user_id": "test_user", "items": [{"sku": "A123", "qty": 1}]}'Observer the Logs:
- Order Service:
202 Accepted - Inventory Service:
Reserved 1 item. Remaining: 99->Published to PaymentQueue - Payment Service:
Payment Success->Order COMPLETED
| Scenario | Input Data | Expected Outcome |
|---|---|---|
| Happy Path | SKU: A123 (Stock: 100) |
Order COMPLETED. Stock: 99. |
| Out of Stock | SKU: B456 (Stock: 0) |
Order FAILED_NO_STOCK. No charge. |
| Payment Fail | Random (10% chance) | Order PAYMENT_FAILED. |
MIT License.