Reference architectures

These reference architectures illustrate how CrateDB can be deployed and integrated in real-world scenarios. They provide guidance on scalability, reliability, data ingestion, and analytics across diverse domains.

1. Real-Time IoT Analytics (Cloud-Native Deployment)

Use Case

Monitoring and analyzing data from a high-volume stream of IoT sensors, such as industrial machinery, smart city infrastructure, or mobility fleets.

Architecture Components

• CrateDB Cloud (multi-node cluster)

• Message Broker: Kafka, MQTT or Azure Event Hub

• Ingestion Layer: Kafka Connect / Fluent Bit

• Real-Time Analytics: CrateDB’s SQL interface + dashboards (e.g., Grafana)

• Downstream Integration: REST APIs or CDC tools (Debezium)

• Security: API keys, TLS, access roles via CrateDB Cloud

2. Edge-to-Cloud Data Pipeline

Use Case

Processing time-series and event data at the edge with periodic sync to a central CrateDB Cloud cluster for advanced analytics.

Architecture Components

• Edge Node: Lightweight CrateDB or TimescaleDB (for local buffering)

• Sync Layer: Scheduled jobs or custom ETL using Python, Kafka, or Debezium

• Central CrateDB Cloud Cluster: Stores harmonized data for analytics

• Analytics/AI Layer: Python, Jupyter, or DB-native aggregations

• Security: Encrypted transfer, RBAC for multi-tenant access

3. Unified Observability Platform

Use Case

Centralizing logs, metrics, and events for DevOps and infrastructure observability.

Architecture Components

• CrateDB Cloud or Self-Hosted Cluster

• Data Sources: Prometheus exporters, Fluent Bit, Logstash, OpenTelemetry

• Ingestion Pipeline: Kafka → CrateDB, with optional transformation layer (e.g., Apache Flink)

• Visualization: Grafana dashboards (via CrateDB plugin)

• Storage Strategy: Partitioned by time and source, TTL policies

• Access Control: SQL RBAC, token-based access for dashboards

4. AI/ML Feature Store Backed by CrateDB

Use Case

Storing and serving high-volume, real-time feature data for machine learning pipelines.

Architecture Components

• CrateDB Cluster: Stores raw data + materialized features

• Data Pipeline: Apache Spark, dbt, or Python ETL for feature computation

• Serving Layer: CrateDB's low-latency SQL queries as REST or gRPC APIs

• Model Training: Python (Pandas, Scikit-learn), notebooks, or Dataiku

• Versioning & Time Travel: Implemented via snapshot tables or time-based partitions

5. Customer 360 and Operational Analytics

Use Case

Combining real-time operational data (clickstreams, events, CRM) with historical records to drive personalized experiences and business insight.

Architecture Components

• Ingestion: Apache Kafka / Confluent → CrateDB

• Batch Ingest: Airbyte, dbt, or custom ETL

• CrateDB Cluster: Serving analytical SQL with fast aggregations and joins

• BI Integration: Tableau, Power BI, or Apache Superset

• API Layer: REST or GraphQL to serve data to apps/portals

6. Multi-Region CrateDB Cloud Deployment

Use Case

High availability and regional data residency compliance across Europe, North America, and Asia.

Architecture Components

• CrateDB Cloud Clusters: One per region

• Global Routing: DNS load balancing + application logic to route by geography

• Data Sync: CDC-based replication, Kafka + Avro schemas, or cross-region export/import jobs

• Failover & DR: Cloud-native replication strategy + backup/restore automation

• Access Control: Project-level isolation, org-level roles

7. CrateDB in Kubernetes (Self-hosted or Hybrid)

Use Case

Deploying CrateDB in Kubernetes for scalability and DevOps automation.

Architecture Components

• Kubernetes Cluster: EKS, AKS, GKE, or on-prem

• CrateDB Helm Chart: Manages StatefulSet + persistent storage

• Operator (optional): For lifecycle automation

• Ingestion: Kafka or REST endpoints

• Monitoring: Prometheus + Grafana with built-in CrateDB metrics

• Security: NetworkPolicies, TLS, Kubernetes secrets

8. Geospatial Analytics Platform

Use Case

Analyzing location-based data (e.g., vehicle fleets, smart infrastructure, geofencing alerts).

Architecture Components

• CrateDB Cluster: With native geospatial functions

• Ingestion: GPS data over REST/Kafka

• Geospatial Indexing: GEO_POINT and GEO_SHAPE columns

• Query Layer: Proximity search, polygon joins, real-time tracking

• Visualization: Kepler.gl, Mapbox, or custom web dashboards

• Alerting: CrateDB scheduled jobs + webhook integration

9. E-commerce Event and Inventory Analytics

Use Case

Tracking product views, carts, conversions, and inventory changes in real time.

Architecture Components

• Frontend Events: Captured via JavaScript SDK → Kafka

• Product and Inventory Data: Synced from ERP/CRM systems

• CrateDB Cluster: Ingests and joins structured + semi-structured data

• Reporting: Ad hoc queries, dashboards, real-time KPIs

• Alerting: Low-stock, anomaly detection with scheduled queries

10. CrateDB + LLM Integration

Use Case

Embedding CrateDB in an AI application stack as a high-performance vector store for RAG (Retrieval-Augmented Generation).

Architecture Components

• LLM Application: LangChain or Haystack

• CrateDB Vector Support: VECTOR data type for embeddings

• Embedding Generation: OpenAI, Hugging Face, etc.

• Similarity Search: KNN queries over VECTOR columns

• Index Management: Background jobs to update embeddings as new data arrives