Temporal Services Guide: Architecture, Clusters, Setup, and Configuration

[anandtk]

Introduction:

A Temporal Cluster or Temporal Service is a distributed system designed for durable execution.

The architecture separates the orchestration logic (the Cluster) from the application logic (Workers).

Core Architecture Components:

A standard production-grade Temporal Cluster setup consists of four primary internal services and external persistence layers:

• Frontend Service: Acts as the gateway for all incoming API calls from clients and workers. It handles rate limiting, authorization, and routing.
• History Service: The core engine that manages workflow state, mutable state, and timers.
• Matching Service: Manages Task Queues and dispatches tasks to available Workers.
• Worker Service: Executes internal system workflows, such as data archival and deletion.
• Database Layer: Stores the actual state. Supported databases include Cassandra, MySQL, and PostgreSQL.
• Visibility Store: Often uses Elasticsearch to enable advanced workflow searching and filtering.

Deployment Models:

• Local Development: Use the Temporal CLI to run a single-binary server (temporal server start-dev) that requires no external dependencies.
• Self-Hosted or Kubernetes: Typically deployed using the official Temporal Helm charts to manage the individual service pods.
• Multi-Cluster Replication: For high availability, Temporal supports asynchronous replication of events across multiple distinct clusters.

External Entities:

The Cluster does not run user code, but it communicates with:

• Application Workers: Hosted by user in thier own VPC or Kubernetes cluster, these processes poll the Matching Service for tasks.
• Temporal Web UI: A console used to inspect namespaces, running workflows, and execution histories.

Setup of Temporal Cluster services or server:

Setting up a Temporal Cluster can be done either for local development (quick setup) or for production (distributed setup).

For development, using Docker Compose is the standard and easiest approach.

For production, it involves deploying components such as the Temporal Server, persistence layer (MySQL/PostgreSQL/Cassandra), and Elasticsearch on platforms like Kubernetes.

Local Development Setup (Docker Compose):

1. Install Prerequisites: Install Docker Desktop and Docker Compose.
2. Clone Repository: Clone the temporalio/docker-compose repository from GitHub.
3. Run Services: Navigate to the docker-compose directory and run docker-compose up to start the Temporal Server, Web UI, and database.
4. Verify: The Web UI is usually accessible at http://localhost:8233

Production Setup (Self-Hosted/Kubernetes):

• Database: Setup a database (PostgreSQL 12+, MySQL 5.7+, or Cassandra).
• Visibility: Use Elasticsearch for advanced visibility features (required for production).
• Components: Deploy the Temporal services: Frontend, History, Matching, and Worker.
• Configuration: Configure dynamic configuration keys, which allow updating settings without restarting.
• Replication: For high availability, set up Multi-Region Namespaces.

Key Considerations:

• Persistence: Choose between Cassandra (best for high throughput) or SQL (Postgres/MySQL).
• Dynamic Config: Update settings like concurrency limits via dynamicconfig without downtime.
• Temporal Cloud: A managed alternative, supporting AWS/GCP and handling infrastructure.

Temporal Service Configuration: It can be done with static and dynamic mode.

Static Configuration:

Static configuration defines how the Temporal Service should be set up. It is read only once at startup and is used to configure the service nodes. Depending on user deployment approach for a self-hosted Temporal Service, the static configuration should include details for setting up:

• Temporal Services—Frontend, History, Matching, Worker.
• Membership ports for the Temporal Services.
• Persistence (including History Shard count), Visibility, Archival store setups.
• TLS, authentication, authorization.
• Server log level.
• Metrics
• Temporal Service metadata
• Dynamic config Client

Temporal Service configuration reference.

https://docs.temporal.io/references/dynamic-configuration

https://github.com/temporalio/temporal/tree/master/config

Dynamic Configuration:

Dynamic configuration consists of configuration keys that can be updated in User's Temporal Service setup without restarting the server processes.

Setting overrides for certain configuration keys takes effect immediately. However, for configuration fields that are only evaluated at startup (such as thread pool size), a server restart is required for the changes to take effect.

Temporal Service configuration reference.

https://docs.temporal.io/references/dynamic-configuration

https://github.com/temporalio/temporal/tree/master/config/dynamicconfig

Also, kindly go through the following my article which provides a detailed explanation of workflow orchestration in Go using Temporal :

Orchestrating Go Workflows with the Temporal Framework