Skip to content

Latest commit

 

History

History
306 lines (236 loc) · 10.7 KB

README.md

File metadata and controls

306 lines (236 loc) · 10.7 KB
SharpCoreDB Logo

SharpCoreDB

High-Performance Embedded Database for .NET 10

License: MIT .NET NuGet Build Tests C#

📌 Current Status — v1.4.1 (March 8, 2026)

✅ Production-Ready: ALL Phase 1-11 Features Complete (100%)

SharpCoreDB v1.4.1 delivers critical bug fixes, 60-80% metadata compression, enterprise-scale distributed features, and a fully functional network database server.

🎉 Major Milestone: All Core Features + Server Complete

Phase 1-11 (100% Complete): SharpCoreDB is now a fully-featured, production-ready embedded AND networked database with advanced analytics, vector search, graph algorithms, distributed capabilities, and server mode.

Latest Achievement: 🚀 Phase 11 - SharpCoreDB.Server COMPLETE (100%)
SharpCoreDB has been successfully transformed from embedded database into a network-accessible database server with gRPC, Binary Protocol, HTTP REST API, and WebSocket streaming support.

Server Features Delivered:

  • ✅ gRPC protocol (HTTP/2 + HTTP/3, primary protocol)
  • ✅ Binary TCP protocol handler
  • ✅ HTTPS REST API (DatabaseController)
  • ✅ WebSocket streaming protocol (real-time query streaming)
  • ✅ JWT authentication + Role-Based Access Control
  • ✅ Mutual TLS (certificate-based authentication)
  • ✅ Multi-database registry with system databases
  • ✅ Connection pooling (1000+ concurrent connections)
  • ✅ Health checks & metrics (Prometheus-compatible)
  • ✅ .NET Client library (ADO.NET-style)
  • ✅ Python client (PySharpDB - published to PyPI)
  • ✅ JavaScript/TypeScript SDK (published to npm)
  • ✅ Docker + Docker Compose deployment
  • ✅ Cross-platform installers (Windows Service, Linux systemd)
  • ✅ Complete server documentation and examples

See documentation: docs/server/PHASE11_IMPLEMENTATION_PLAN.md

🎯 Latest Release (v1.4.0 → v1.4.1)

  • 🐛 Critical Bug Fixes

    • Database reopen edge case fixed (graceful empty JSON handling)
    • Immediate metadata flush ensures durability
    • Enhanced error messages with JSON preview

  • 📦 New Features

    • Brotli compression for JSON metadata (60-80% size reduction)
    • Backward compatible format detection
    • Zero breaking changes

  • 📊 Quality Metrics

    • 1,468+ tests (was 850+ in v1.3.5)
    • 100% backward compatible
    • All 11 phases production-ready

🚀 Complete Feature Set (Phases 1-11)

Phase 11: SharpCoreDB.Server (Network Database Server) ✅

  • gRPC protocol (HTTP/2 + HTTP/3) - primary, high-performance protocol
  • Binary TCP protocol - PostgreSQL wire protocol compatibility
  • HTTPS REST API - web browser and simple integration support
  • WebSocket streaming - real-time query streaming
  • JWT + Mutual TLS authentication
  • Role-Based Access Control (Admin/Writer/Reader)
  • Multi-database support with system databases
  • Connection pooling (1000+ concurrent connections)
  • Health checks & Prometheus metrics
  • .NET, Python, JavaScript/TypeScript client libraries
  • Docker + cross-platform installers (Windows/Linux)
  • Complete documentation and examples

Phase 10: Enterprise Distributed Features ✅

  • Multi-master replication with vector clocks (Phase 10.2)
  • Distributed transactions with 2PC protocol (Phase 10.3)
  • Dotmim.Sync integration for cloud sync (Phase 10.1)

Phase 9: Advanced Analytics ✅

  • 100+ aggregate functions (COUNT, SUM, AVG, STDDEV, VARIANCE, PERCENTILE, CORRELATION)
  • Window functions (ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD)
  • 150-680x faster than SQLite

Phase 8: Vector Search ✅

  • HNSW indexing with SIMD acceleration
  • 50-100x faster than SQLite
  • Production-tested with 10M+ vectors

Phase 6: Graph Algorithms ✅

  • A* pathfinding (30-50% improvement)
  • Graph traversal (BFS, DFS, bidirectional search)
  • ROWREF data type for graph edges
  • GRAPH_TRAVERSE() SQL function

Phases 1-5: Core Engine ✅

  • Single-file encrypted database (AES-256-GCM)
  • SQL support with advanced query optimization
  • ACID transactions with WAL
  • B-tree and hash indexing
  • Full-text search
  • SIMD-accelerated operations
  • Memory pooling and JIT optimizations

📦 Installation

# Core database (v1.4.1 - NOW WITH METADATA COMPRESSION!)
dotnet add package SharpCoreDB --version 1.4.1

# Server mode (network database server with gRPC/HTTP/WebSocket)
dotnet add package SharpCoreDB.Server --version 1.4.1
dotnet add package SharpCoreDB.Client --version 1.4.1

# Distributed features (multi-master replication, 2PC transactions)
dotnet add package SharpCoreDB.Distributed --version 1.4.1

# Analytics engine (100+ aggregate & window functions)
dotnet add package SharpCoreDB.Analytics --version 1.4.1

# Vector search (HNSW indexing, semantic search)
dotnet add package SharpCoreDB.VectorSearch --version 1.4.1

# Sync integration (bidirectional sync with SQL Server/PostgreSQL/MySQL/SQLite)
dotnet add package SharpCoreDB.Provider.Sync --version 1.4.1

# Graph algorithms (A* pathfinding)
dotnet add package SharpCoreDB.Graph --version 1.4.1

# Optional integrations
dotnet add package SharpCoreDB.EntityFrameworkCore --version 1.4.1
dotnet add package SharpCoreDB.Extensions --version 1.4.1
dotnet add package SharpCoreDB.Serilog.Sinks --version 1.4.1

# Client libraries for other languages
# Python: pip install pysharpcoredb
# JavaScript/TypeScript: npm install @sharpcoredb/client

🚀 Performance Benchmarks

Operation SharpCoreDB SQLite Delta
Bulk Insert (1M rows) 2.8s 18.2s 6.5x faster
COUNT (1M rows) 0.8ms 544ms 682x faster
Window Functions 15ms 2.3s 156x faster
Vector Search (10M) 1.2ms 120ms 100x faster
Metadata Compression 24KB → 5.8KB N/A 75% reduction
gRPC Query Latency 0.8-1.2ms N/A Sub-millisecond
Concurrent Connections 1000+ N/A Server mode

🎯 Core Features

✅ Production-Ready Capabilities

  • Single-file encrypted database with AES-256-GCM
  • Full SQL support with advanced query optimization
  • ACID transactions with Write-Ahead Logging (WAL)
  • Multi-version concurrency control (MVCC)
  • Automatic indexing (B-tree and hash)

🌐 Network Database Server (NEW in Phase 11)

  • gRPC Protocol - Primary, high-performance protocol (HTTP/2 + HTTP/3)
  • Binary TCP Protocol - PostgreSQL wire protocol compatibility
  • HTTPS REST API - Web browser and simple integration support
  • WebSocket Streaming - Real-time query streaming
  • Enterprise Security - JWT + Mutual TLS authentication, RBAC
  • Connection Pooling - 1000+ concurrent connections
  • Multi-Database Support - Multiple databases + system databases
  • Health & Metrics - Prometheus-compatible monitoring
  • Cross-Platform - Docker, Windows Service, Linux systemd
  • Client Libraries - .NET (ADO.NET-style), Python (PyPI), JavaScript/TypeScript (npm)

📊 Analytics & Data Processing

  • 100+ aggregate functions
  • Window functions for complex analysis
  • Statistical analysis (STDDEV, VARIANCE, PERCENTILE, CORRELATION)
  • 150-680x faster than SQLite for analytics

🔍 Vector & Semantic Search

  • HNSW indexing with SIMD acceleration
  • Semantic similarity search
  • 50-100x faster than SQLite
  • Production-tested with 10M+ vectors

🌐 Enterprise Distributed Features

  • Multi-master replication across nodes
  • Distributed transactions with 2PC protocol
  • Bidirectional sync with cloud databases
  • Automatic conflict resolution
  • Vector clock-based causality tracking

📱 Cross-Platform Support

  • Windows (x64, ARM64)
  • Linux (x64, ARM64)
  • macOS (x64, ARM64)
  • Android, iOS (via portable library)
  • IoT/Embedded devices
  • Docker - Official container images
  • Cloud - Azure, AWS, GCP compatible

💻 Quick Start

Embedded Mode

using SharpCoreDB;

// Create encrypted database
var factory = new DatabaseFactory();
var db = factory.Create("myapp.scdb", "master-password");

// Create table and insert data
db.ExecuteSQL("CREATE TABLE users (id INT PRIMARY KEY, name TEXT)");
db.ExecuteSQL("INSERT INTO users VALUES (1, 'Alice')");

// Query with advanced analytics
var results = db.ExecuteQuery(
  "SELECT name, COUNT(*) as count FROM users GROUP BY name"
);

// Persist to disk
db.Flush();

Server Mode (NEW!)

Start the Server:

# Using Docker
docker run -d -p 5001:5001 -p 8443:8443 \
  -v /data/sharpcoredb:/data \
  sharpcoredb/server:latest

# Or using Windows Service
sc create SharpCoreDBServer binPath="C:\Program Files\SharpCoreDB\SharpCoreDB.Server.exe"
sc start SharpCoreDBServer

# Or using Linux systemd
sudo systemctl enable sharpcoredb
sudo systemctl start sharpcoredb

Connect from .NET:

using SharpCoreDB.Client;

// Connect to SharpCoreDB server
await using var connection = new SharpCoreDBConnection(
    "Server=localhost;Port=5001;Database=mydb;SSL=true;Username=admin;Password=***"
);
await connection.OpenAsync();

// Execute queries
await using var command = new SharpCoreDBCommand("SELECT * FROM users WHERE age > @age", connection);
command.Parameters.Add("@age", 21);

await using var reader = await command.ExecuteReaderAsync();
while (await reader.ReadAsync())
{
    Console.WriteLine($"Name: {reader["name"]}, Age: {reader["age"]}");
}

Connect from Python:

import asyncio
import pysharpcoredb as scdb

async def main():
    async with scdb.connect("grpc://localhost:5001", database="mydb") as conn:
        result = await conn.execute("SELECT * FROM users WHERE age > ?", {"age": 21})
        print(f"Found {len(result)} users")

asyncio.run(main())

Connect from JavaScript/TypeScript:

import { connect } from '@sharpcoredb/client';

const connection = await connect('grpc://localhost:5001', { database: 'mydb' });
const result = await connection.execute('SELECT * FROM users WHERE age > ?', { age: 21 });
console.log(`Found ${result.rows.length} users`);
await connection.close();