Securing Temporal with Keycloak: Implementing Custom Role-Based Access Control

[Manjunatha-KJ]

Introduction & Problem Statement

As distributed architectures scale, workflow orchestration platforms like Temporal rapidly become the mission-critical nervous system of the enterprise. By default, Temporal offers powerful workflow execution capabilities, but in a highly regulated enterprise environment, relying on basic authentication or perimeter security isn't enough.

The engineering team faced a significant problem: How do we enforce strict, multi-tenant isolation and granular Role-Based Access Control (RBAC) using our central Identity Provider (Keycloak), without sacrificing performance or forking the entire Temporal codebase? This post explores the journey of integrating Keycloak directly into the Temporal gRPC layer, ensuring that every workflow execution, signal, and query is cryptographically verified and strictly authorized.

Why RBAC Matters in Temporal

Temporal's architecture is highly transparent, meaning any client with network access to the frontend service can potentially start, query, or terminate workflows. Without granular RBAC, the blast radius of a compromised service or an accidental script is cluster wide.

Implementing RBAC natively at the gRPC layer ensures a zero-trust boundary. It guarantees that a microservice can only interact with the workflows and namespaces it explicitly owns and can only perform actions (like signalling or terminating) that its assigned role permits.

Real-World Enterprise Use Case

Imagine a scenario where the Human Resources team's onboarding workflows and the Finance team's payroll processing workflows reside on the same Temporal cluster.

Without strict namespace isolation and role checks, an errant script in the HR domain could potentially query sensitive financial workflows or, worse, send a termination signal to a payroll process. By implementing custom RBAC, the platform securely segregates these domains. A developer debugging HR workflows is granted a workflow-operator role restricted entirely to the HR namespace, ensuring complete isolation from Finance.

Architecture Overview

Rather than building a brittle proxy layer in front of Temporal, the team leveraged Temporal's extensible architecture. By importing Temporal as a Go library, the team injected custom authorization plugins and compiled a bespoke temporal-server binary.

The architecture intercepts every request to the Temporal server, validates an incoming Keycloak JSON Web Token (JWT), and evaluates the identity's permissions against the requested action.

Token Propagation Design

Security begins at the client edge. When a frontend application or external service makes an HTTP request to the internal backend APIs, it includes a Keycloak JWT in the Authorization header.

The backend API extracts this token and injects it into the Go context.Context. To ensure this token reaches Temporal, a custom TokenHeadersProvider is attached to the Temporal client SDK. This provider automatically extracts the token from the context and attaches it as gRPC metadata to every outgoing call made to the Temporal cluster, ensuring the identity is propagated securely across network boundaries.

Custom Claim Mapper

The ClaimMapper interface acts as the bridge between Keycloak and Temporal. It intercepts the gRPC request, validates the JWT signature against the Keycloak JWKS endpoint, and translates Keycloak's realm_access roles into Temporal's internal roles (RoleAdmin, RoleWriter, RoleReader).

Custom Authorizer

With claims mapped, the Authorizer evaluates the request. To solve the business problem of restricting state-mutating actions, the logic explicitly intercepts and guards specific API calls, such as signals.

End-to-End Request Flow

When an HTTP request triggers a workflow signal, the full lifecycle looks like this:

1. Client: A service calls the backend API with a valid Keycloak JWT.
2. Propagation: The backend Temporal Client attaches the JWT to the gRPC metadata.
3. Validation: Temporal receives the gRPC call; the ClaimMapper intercepts the JWT and validates its signature against Keycloak's public keys.
4. Mapping: The ClaimMapper sees the workflow-operator role and maps it to RoleWriter for the target namespace.
5. Authorization: The Authorizer checks the requested API (SignalWorkflowExecution). Since the identity holds RoleWriter, the action is permitted.
6. Execution: The workflow is successfully signaled.

Testing and Validation

Testing authorization logic requires simulating both authorized and unauthorized identities.

During development, the team utilized a local Go runner (go run main.go auth.go) combined with a mock Identity Provider. Integration tests were written to forcefully inject expired tokens, tokens with missing roles, and tokens with incorrect signatures to verify that the Temporal server correctly instantly rejected the gRPC calls with PermissionDenied errors.

Production Considerations

Deploying a custom Temporal binary into production involves specific operational strategies. By using a multi-stage Dockerfile, the custom Go binary is compiled and subsequently injected into the official temporalio/server base image.

This ensures that the deployment retains all official startup scripts, database schemas, and tooling. When deploying via Kubernetes, the image directive simply points to the custom registry image, and the Keycloak JWKS URL is provided dynamically via environment variables (KEYCLOAK_JWKS_URL), ensuring seamless promotion across staging and production environments.

Lessons Learned

Building native Temporal RBAC came with valuable insights:

1. Caching is Critical: The ClaimMapper must validate the JWT signature using Keycloak's public keys. Fetching the JWKS on every gRPC call would cripple performance. Utilizing a robust JWKS caching library is non-negotiable.
2. Debugging Interceptors: When authorization fails at the gRPC level, the client simply sees a generic PermissionDenied error. Emitting highly detailed, structured logs inside the ClaimMapper and Authorizer proved essential for diagnosing token expiry or role misconfigurations.
3. Backward Compatibility: It is crucial to ensure that internal Temporal system calls (e.g., to the temporal-system namespace) bypass the standard authorization checks to prevent the cluster from locking itself out of its own internal workflows.

Conclusion

By treating Temporal as a flexible framework rather than a black box, the engineering team successfully transformed a standard orchestration engine into a secure, multi-tenant platform.

The gRPC layer now acts as an intelligent, zero-trust boundary, perfectly aligned with the enterprise's central identity policies. For any team scaling Temporal across multiple domains, investing in custom RBAC via the ClaimMapper and Authorizer is a foundational step toward mature, secure orchestration.

[olio35xender]

Hello,
This is a great deep dive into securing Temporal for enterprise environments. I especially like the decision to integrate authorization directly into the gRPC layer instead of relying on a proxy, as it keeps security close to the execution path while avoiding unnecessary complexity. The Keycloak JWT validation, custom claim mapping, and context-based token propagation provide a clean zero-trust implementation. The HR vs. Finance example clearly demonstrates the practical value of namespace isolation and granular RBAC. I'd also be interested in learning how you handle JWT caching, token refresh, and performance under high workflow throughput, as those are often critical considerations in production deployments.


Best Regards

[Manjunatha-KJ]

Hi olio,

Thanks for your comment! I'm happy you found the post useful. You're right, adding authorization at the gRPC layer has some clear advantages, but things like token management become important once you're running this in production. Here's the general approach we follow:

---

1. JWT Validation and Caching

- We don't call the identity provider(keycloak) to validate every token because that would add unnecessary network calls and slow things down.

- Instead, we validate JWTs locally using the public keys (JWKS). Those keys are cached in memory, so most token validations happen locally and are very fast.

- If the identity provider rotates its keys, we refresh the cache periodically or whenever we see a new key ID that isn't already cached.

2. Token Refresh

- Since Temporal workflows and workers can run for a long time, access tokens will eventually expire.

- Rather than waiting for a request to fail, we keep track of the token's expiry time. If it's getting close to expiring, a background process fetches a new token and updates the cached one. This happens automatically, so the application keeps running without interruptions.

- If a request still fails because of an expired token, the retry mechanism can retry it after a fresh token is available.

3. Performance at Scale

- When you're processing a large number of workflows, the goal is to keep the authorization checks lightweight.

- Because token validation happens locally, there are no extra network calls for every request. Signature verification is also fast, so the additional CPU cost is usually small.

- If role or permission checks are needed, we keep that logic simple and efficient instead of doing expensive processing for every request.

- We also use rate limiting to protect the service during sudden traffic spikes.

---

It takes a little work to set up the caching and refresh logic properly, but once it's in place, it performs well and scales without adding much overhead.

Thanks again for reading and for the great questions!