From 4fa57ab2c44557fb961e9bbac41c3c5b16d466dc Mon Sep 17 00:00:00 2001 From: Andres Contreras Date: Wed, 11 Feb 2026 10:22:31 +0100 Subject: [PATCH] docs: add professional README with unified template (#7) Standardize README.md with a consistent template including CI badge, features, installation, quick start, configuration, and documentation sections. Add paths-ignore to ci.yml to skip CI for docs-only changes. --- .github/workflows/ci.yml | 19 +- README.md | 483 +++++++-------------------------------- 2 files changed, 101 insertions(+), 401 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b9093b6..9567aee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,8 +2,22 @@ name: CI on: push: branches: [develop] + paths-ignore: + - '**.md' + - 'docs/**' + - 'tutorials/**' + - 'examples/**/README.md' + - 'LICENSE' + - '.gitignore' pull_request: branches: [develop, main] + paths-ignore: + - '**.md' + - 'docs/**' + - 'tutorials/**' + - 'examples/**/README.md' + - 'LICENSE' + - '.gitignore' workflow_dispatch: inputs: triggered-by: @@ -13,10 +27,5 @@ on: jobs: build: uses: fireflyframework/.github/.github/workflows/java-ci.yml@main - permissions: - packages: read - contents: read - actions: write with: java-version: '25' - secrets: inherit diff --git a/README.md b/README.md index f250908..e878fe1 100644 --- a/README.md +++ b/README.md @@ -1,441 +1,132 @@ -# Firefly Framework Callbacks Library - -[![CI](https://github.com/fireflyframework/fireflyframework-callbacks/actions/workflows/ci.yml/badge.svg)](https://github.com/fireflyframework/fireflyframework-callbacks/actions/workflows/ci.yml) +# Firefly Framework - Callbacks +[![CI](https://github.com/fireflyframework/fireflyframework-callbacks/actions/workflows/ci.yml/badge.svg)](https://github.com/fireflyframework/fireflyframework-callbacks/actions/workflows/ci.yml) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) -[![Java](https://img.shields.io/badge/Java-25-orange.svg)](https://openjdk.org/projects/jdk/21/) -[![Spring Boot](https://img.shields.io/badge/Spring%20Boot-3.2.2-brightgreen.svg)](https://spring.io/projects/spring-boot) -[![Maven](https://img.shields.io/badge/Maven-3.8+-red.svg)](https://maven.apache.org/) +[![Java](https://img.shields.io/badge/Java-21%2B-orange.svg)](https://openjdk.org) +[![Spring Boot](https://img.shields.io/badge/Spring%20Boot-3.x-green.svg)](https://spring.io/projects/spring-boot) -> **Enterprise-grade outbound webhook management platform for sending real-time events from Firefly to third-party systems** +> Outbound callback management platform for dispatching events to external systems with circuit breakers, retry logic, and domain authorization. -## πŸ“‹ Table of Contents +--- + +## Table of Contents - [Overview](#overview) -- [Key Features](#key-features) -- [Architecture](#architecture) +- [Features](#features) +- [Requirements](#requirements) +- [Installation](#installation) - [Quick Start](#quick-start) -- [Documentation](#documentation) -- [Technology Stack](#technology-stack) -- [Project Structure](#project-structure) -- [API Endpoints](#api-endpoints) - [Configuration](#configuration) -- [Building & Running](#building--running) -- [Testing](#testing) +- [Documentation](#documentation) +- [Contributing](#contributing) - [License](#license) -## 🎯 Overview - -The **Firefly Framework Callbacks Library** is a reactive, event-driven microservice that enables Firefly to send real-time webhooks to external systems. It provides a complete solution for: - -- **Dynamic Event Subscription**: Subscribe to Kafka topics at runtime without redeployment -- **Intelligent Event Routing**: Route events to multiple HTTP endpoints based on event types -- **Reliable Delivery**: Circuit breakers, exponential backoff retries, and execution tracking -- **Security First**: Domain authorization, HMAC signatures, and HTTPS enforcement -- **Multi-Tenant**: Full tenant isolation and configuration -- **Production Ready**: Comprehensive monitoring, metrics, and observability - -### Use Cases - -- **CRM Integration**: Send customer lifecycle events to Salesforce, HubSpot, or custom CRMs -- **Analytics**: Stream events to data warehouses or analytics platforms -- **Notifications**: Trigger external notification systems on business events -- **Workflow Automation**: Integrate with Zapier, Make, or custom automation platforms -- **Audit & Compliance**: Forward events to external audit systems - -## ✨ Key Features - -### πŸ”„ Dynamic Event Subscription -- Subscribe to Kafka topics through REST API -- No code changes or redeployment required -- Support for event type patterns (`customer.*`, `order.completed`) -- Automatic listener registration via `lib-common-eda` - -### 🎯 Intelligent Routing -- Route events to multiple HTTP endpoints -- Filter expressions for fine-grained control -- Event type matching with wildcard support -- Parallel callback execution - -### πŸ›‘οΈ Enterprise Security -- **Domain Authorization**: Whitelist-based callback URL validation -- **HMAC Signatures**: SHA-256 signed payloads for webhook verification -- **HTTPS Enforcement**: Configurable per domain -- **Path Restrictions**: Control allowed URL paths -- **Rate Limiting**: Per-domain callback rate limits - -### πŸ” Reliability & Resilience -- **Circuit Breaker**: Per-configuration isolation using Resilience4j -- **Exponential Backoff**: Configurable retry delays with multipliers -- **Timeout Management**: Per-callback timeout configuration -- **Execution Tracking**: Complete audit trail of all callback attempts - -### πŸ“Š Observability -- **Prometheus Metrics**: Circuit breaker states, retry counts, execution times -- **Spring Boot Actuator**: Health checks, info endpoints -- **Structured Logging**: JSON-formatted logs with correlation IDs -- **Execution History**: Queryable database of all callback attempts - -### πŸš€ Performance -- **Reactive Architecture**: Built on Spring WebFlux and Project Reactor -- **Non-Blocking I/O**: R2DBC for database, WebClient for HTTP -- **Connection Pooling**: Optimized database and HTTP client pools -- **Async Processing**: Event-driven, non-blocking execution - -## πŸ—οΈ Architecture +## Overview -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Firefly Servicesβ”‚ -β”‚ (Customer, β”‚ -β”‚ Loan, etc.) β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ - β”‚ Publish Events - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Apache Kafka β”‚ -β”‚ (Topics) β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ - β”‚ Consume - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Callback Management Platform β”‚ -β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Dynamic │───▢│ Callback β”‚ β”‚ -β”‚ β”‚ Listener β”‚ β”‚ Router β”‚ β”‚ -β”‚ β”‚ Factory β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β–Ό β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Dispatcher β”‚ β”‚ -β”‚ β”‚ (Circuit β”‚ β”‚ -β”‚ β”‚ Breaker + β”‚ β”‚ -β”‚ β”‚ Retry) β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Domain │◀─────────── β”‚ Execution β”‚ β”‚ -β”‚ β”‚Authorization β”‚ β”‚ β”‚ Tracker β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ -β”‚ β–Ό β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ PostgreSQL β”‚ β”‚ -β”‚ β”‚ (R2DBC) β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ - β”‚ HTTP Callbacks - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Third-Party β”‚ -β”‚ Systems β”‚ -β”‚ (CRM, Analytics,β”‚ -β”‚ Webhooks, etc.)β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` +Firefly Framework Callbacks provides an outbound webhook/callback management system for dispatching events to external systems. It enables applications to register callback configurations, subscribe to events, manage authorized domains, and track callback execution history with comprehensive retry and circuit breaker support. -**Event Flow:** -1. Firefly services publish events to Kafka topics -2. Dynamic listeners consume events based on database subscriptions -3. Callback router finds matching configurations -4. Domain authorization validates callback URLs -5. Dispatcher executes HTTP callbacks with retry/circuit breaker -6. Execution results are tracked in PostgreSQL +The project is structured as a multi-module build with five sub-modules: interfaces (DTOs and enums), models (database entities and repositories), core (services and business logic), SDK (client library), and web (REST controllers). The core module includes callback dispatching with routing, domain authorization, and event subscription management. -For detailed architecture documentation, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). +The callback dispatcher handles reliable event delivery to registered endpoints with configurable retry policies, execution tracking, and authorized domain validation for security. -## πŸš€ Quick Start +## Features -### Prerequisites +- Callback configuration management with CRUD operations +- Event subscription system for selective event routing +- Authorized domain management for endpoint security +- Callback dispatcher with reliable event delivery +- Callback router for multi-endpoint event distribution +- Dynamic listener factory for runtime event binding +- Execution tracking with status history (pending, success, failed) +- Filtering and pagination for configuration and execution queries +- Domain authorization service for endpoint validation +- REST controllers for callbacks, subscriptions, domains, and executions +- Multi-module architecture: interfaces, models, core, SDK, web -- **Java 25** (default, Java 21+ compatible) -- **Maven 3.8+** -- **PostgreSQL 14+** -- **Apache Kafka 3.0+** (or Confluent Platform 7.5+) -- **Docker** (optional, for running dependencies) +## Requirements -### 1. Clone the Repository +- Java 21+ +- Spring Boot 3.x +- Maven 3.9+ +- PostgreSQL database (for callback persistence) -```bash -git clone https://github.com/firefly-oss/fireflyframework-callbacks.git -cd fireflyframework-callbacks -``` +## Installation -### 2. Start Dependencies (Docker) +The callbacks library is a multi-module project. Include the modules you need: -```bash -# Start PostgreSQL and Kafka -docker-compose up -d postgres kafka -``` +```xml + + + org.fireflyframework + fireflyframework-callbacks-core + 26.01.01 + -### 3. Configure Environment + + + org.fireflyframework + fireflyframework-callbacks-interfaces + 26.01.01 + -```bash -export DB_HOST=localhost -export DB_PORT=5432 -export DB_NAME=callbacks_db -export DB_USERNAME=firefly -export DB_PASSWORD=firefly -export KAFKA_BOOTSTRAP_SERVERS=localhost:9092 + + + org.fireflyframework + fireflyframework-callbacks-sdk + 26.01.01 + ``` -### 4. Build the Project +## Quick Start -```bash -mvn clean install -``` - -### 5. Run the Application - -```bash -cd fireflyframework-callbacks-web -mvn spring-boot:run -``` - -The application will start on `http://localhost:8080`. +```java +import org.fireflyframework.callbacks.core.service.CallbackDispatcher; -### 6. Verify Installation +@Service +public class OrderEventPublisher { -```bash -# Check health -curl http://localhost:8080/actuator/health + private final CallbackDispatcher callbackDispatcher; -# Access Swagger UI -open http://localhost:8080/swagger-ui.html + public Mono publishOrderCreated(Order order) { + return callbackDispatcher.dispatch("order.created", order); + } +} ``` -For a complete quickstart guide, see [docs/QUICKSTART_GUIDE.md](docs/QUICKSTART_GUIDE.md). - -## πŸ“š Documentation - -| Document | Description | -|----------|-------------| -| [**Callback System Summary**](docs/CALLBACK_SYSTEM_SUMMARY.md) | **⭐ START HERE** - Executive summary of the complete callback system implementation | -| [Callback System Reference](docs/CALLBACK_SYSTEM_REFERENCE.md) | Complete reference - domains, configurations, HMAC, retries, metadata, best practices | -| [Callback Examples](docs/CALLBACK_EXAMPLES.md) | Practical examples - Salesforce, Zapier, OAuth, custom integrations, testing | -| [Architecture Deep Dive](docs/ARCHITECTURE.md) | Complete architectural overview, design patterns, and data models | -| [Quickstart Guide](docs/QUICKSTART_GUIDE.md) | Step-by-step guide to get started quickly | -| [Testing Guide](docs/TESTING_GUIDE.md) | Comprehensive testing strategies and examples | - -## πŸ› οΈ Technology Stack - -### Core Framework -- **Spring Boot 3.5.10** - Application framework -- **Spring WebFlux** - Reactive web framework -- **Project Reactor** - Reactive programming library - -### Data Layer -- **Spring Data R2DBC** - Reactive database access -- **PostgreSQL** - Primary database -- **Flyway** - Database migrations - -### Messaging -- **Apache Kafka** - Event streaming platform -- **lib-common-eda** - Firefly's event-driven architecture library - -### Resilience -- **Resilience4j** - Circuit breaker and retry patterns -- **Spring Retry** - Declarative retry support - -### Observability -- **Spring Boot Actuator** - Production-ready features -- **Micrometer** - Metrics facade -- **Prometheus** - Metrics collection - -### Development -- **Lombok** - Boilerplate reduction -- **MapStruct** - DTO-Entity mapping -- **OpenAPI 3** - API documentation - -### Testing -- **JUnit 5** - Testing framework -- **Testcontainers** - Integration testing with Docker -- **WireMock** - HTTP mocking -- **Reactor Test** - Reactive testing utilities - -## πŸ“ Project Structure - -``` -fireflyframework-callbacks/ -β”œβ”€β”€ fireflyframework-callbacks-interfaces/ # DTOs, Enums, API contracts -β”‚ └── src/main/java/.../interfaces/ -β”‚ β”œβ”€β”€ dto/ # Data Transfer Objects -β”‚ └── enums/ # Enumerations -β”‚ -β”œβ”€β”€ fireflyframework-callbacks-models/ # Entities, Repositories -β”‚ └── src/main/ -β”‚ β”œβ”€β”€ java/.../models/ -β”‚ β”‚ β”œβ”€β”€ entity/ # R2DBC entities -β”‚ β”‚ └── repository/ # R2DBC repositories -β”‚ └── resources/db/migration/ # Flyway migrations -β”‚ -β”œβ”€β”€ fireflyframework-callbacks-core/ # Business logic -β”‚ └── src/main/java/.../core/ -β”‚ β”œβ”€β”€ filters/ # FilterRequest, FilterUtils -β”‚ β”œβ”€β”€ listener/ # Dynamic event listeners -β”‚ β”œβ”€β”€ mapper/ # MapStruct mappers -β”‚ └── service/ # Service interfaces & implementations -β”‚ -β”œβ”€β”€ fireflyframework-callbacks-web/ # REST API, Configuration -β”‚ └── src/main/ -β”‚ β”œβ”€β”€ java/.../web/ -β”‚ β”‚ β”œβ”€β”€ config/ # Spring configuration -β”‚ β”‚ └── controller/ # REST controllers -β”‚ └── resources/ -β”‚ └── application.yml # Application configuration -β”‚ -└── fireflyframework-callbacks-sdk/ # Client SDK (future) -``` - -## πŸ”Œ API Endpoints - -### Event Subscriptions -``` -POST /api/v1/event-subscriptions/filter # Filter subscriptions (paginated) -POST /api/v1/event-subscriptions # Create subscription -GET /api/v1/event-subscriptions/{id} # Get by ID -PUT /api/v1/event-subscriptions/{id} # Update subscription -DELETE /api/v1/event-subscriptions/{id} # Delete subscription -``` - -### Authorized Domains -``` -POST /api/v1/authorized-domains/filter # Filter domains (paginated) -POST /api/v1/authorized-domains # Create domain -PUT /api/v1/authorized-domains/{id} # Update domain -POST /api/v1/authorized-domains/{domain}/verify # Verify domain (requires verificationMethod param) -DELETE /api/v1/authorized-domains/{id} # Delete domain (requires domain param) -``` - -### Callback Configurations -``` -POST /api/v1/callback-configurations/filter # Filter configurations (paginated) -POST /api/v1/callback-configurations # Create configuration -GET /api/v1/callback-configurations/{id} # Get by ID -PUT /api/v1/callback-configurations/{id} # Update configuration -DELETE /api/v1/callback-configurations/{id} # Delete configuration -``` - -### Callback Executions -``` -POST /api/v1/callback-executions/filter # Filter executions (paginated) -GET /api/v1/callback-executions # List all executions -GET /api/v1/callback-executions/{id} # Get by ID -GET /api/v1/callback-executions/by-configuration/{configId} # List by configuration -GET /api/v1/callback-executions/by-status?status={status} # List by status -GET /api/v1/callback-executions/pending-retries # List pending retries -GET /api/v1/callback-executions/recent?configurationId={id}&duration={duration} # List recent executions -``` - -**Full API documentation available at:** `http://localhost:8080/swagger-ui.html` - -## βš™οΈ Configuration - -Key configuration properties in `application.yml`: +## Configuration ```yaml -# Database -spring: - r2dbc: - url: r2dbc:postgresql://${DB_HOST:localhost}:${DB_PORT:5432}/${DB_NAME:callbacks_db} - username: ${DB_USERNAME:firefly} - password: ${DB_PASSWORD:firefly} - -# Kafka (via Firefly EDA) firefly: - eda: - consumer: - enabled: true - kafka: - default: - bootstrap-servers: ${FIREFLY_KAFKA_BOOTSTRAP_SERVERS} - properties: - group.id: callbacks-mgmt-consumer - - # Callbacks Configuration callbacks: - circuit-breaker: - failure-rate-threshold: ${CALLBACK_CB_FAILURE_RATE:50} - wait-duration-in-open-state-ms: ${CALLBACK_CB_WAIT_DURATION:60000} - retry: - max-attempts: ${CALLBACK_RETRY_MAX_ATTEMPTS:3} - initial-delay-ms: ${CALLBACK_RETRY_INITIAL_DELAY:1000} - max-delay-ms: ${CALLBACK_RETRY_MAX_DELAY:60000} -``` - -See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for complete configuration reference. - -## πŸ”¨ Building & Running - -### Build - -```bash -# Build all modules -mvn clean install - -# Skip tests -mvn clean install -DskipTests - -# Build specific module -mvn clean install -pl fireflyframework-callbacks-web -``` - -### Run - -```bash -# Run with Maven -cd fireflyframework-callbacks-web -mvn spring-boot:run - -# Run with Java -java -jar fireflyframework-callbacks-web/target/fireflyframework-callbacks-web-1.0.0-SNAPSHOT.jar - -# Run with Docker -docker build -t firefly/callbacks-mgmt:latest . -docker run -p 8080:8080 firefly/callbacks-mgmt:latest -``` - -## πŸ§ͺ Testing - -```bash -# Run all tests -mvn test - -# Run specific test class -mvn test -Dtest=CallbackManagementEndToEndTest - -# Run integration tests only -mvn verify -P integration-tests + max-attempts: 3 + backoff-multiplier: 2 + initial-delay: 1s + domain-authorization: + enabled: true -# Generate coverage report -mvn clean test jacoco:report +spring: + r2dbc: + url: r2dbc:postgresql://localhost:5432/callbacks ``` -**Test Coverage:** -- Unit Tests: Service layer, mappers, utilities -- Integration Tests: Repository layer with Testcontainers -- End-to-End Tests: Full flow with Kafka, PostgreSQL, and WireMock +## Documentation -See [docs/TESTING_GUIDE.md](docs/TESTING_GUIDE.md) for detailed testing documentation. +Additional documentation is available in the [docs/](docs/) directory: -## πŸ“„ License +- [Quickstart Guide](docs/QUICKSTART_GUIDE.md) +- [Architecture](docs/ARCHITECTURE.md) +- [Callback System Reference](docs/CALLBACK_SYSTEM_REFERENCE.md) +- [Callback System Summary](docs/CALLBACK_SYSTEM_SUMMARY.md) +- [Callback Examples](docs/CALLBACK_EXAMPLES.md) +- [Testing Guide](docs/TESTING_GUIDE.md) -**Β© 2025 Firefly Software Solutions Inc. All rights reserved.** +## Contributing -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at +Contributions are welcome. Please read the [CONTRIBUTING.md](CONTRIBUTING.md) guide for details on our code of conduct, development process, and how to submit pull requests. - http://www.apache.org/licenses/LICENSE-2.0 +## License -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. - ---- +Copyright 2024-2026 Firefly Software Solutions Inc. -**Developed with ❀️ by the Firefly Team** \ No newline at end of file +Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details.