API Bridge : A Declarative API Orchestration Engine

[Manjunatha-KJ]

API Bridge : A Declarative API Orchestration Engine

---

A customer dashboard may need profile, order, loyalty, and recommendation data; a loan application may require identity checks, account creation, bank validation, and disbursement. When every caller coordinates these requests independently, orchestration logic spreads across the system. A declarative orchestration engine offers another option: define reusable backend workflows as configuration and let a dedicated service execute them.

This does not make Backend-for-Frontend (BFF) services obsolete. A BFF can still handle client-specific presentation and authentication, while an orchestration engine owns reusable backend integration flows.

---

Why API Composition Gets Difficult

---

A single user action may depend on several independently deployed services. If each client coordinates those calls, teams can encounter:

1. Extra network round trips: Sequential calls add latency, especially for mobile or high-latency clients.
2. Duplicated BFF logic: BFFs are useful for client-specific needs, but reusable integration rules can become scattered across multiple services.
3. State-passing boilerplate: One service's response often becomes another service's request, requiring repeated mapping code.
4. Inconsistent failure handling: Timeouts and partial failures can be handled differently by every caller.
5. Configuration drift: Similar workflows evolve separately, making behavior harder to audit and update consistently.

The goal is not to remove all client-side or BFF responsibilities. It is to move reusable backend sequencing and data mapping into one place where it can be inspected and managed.

---

The Idea: Describe the Flow, Keep the Client Simple

---

Instead of hard-coding every sequence of API calls, teams register a workflow under a name and trigger it with live request data. The engine looks up the workflow, maps request values into downstream calls, runs the configured steps, and returns a result.

For changes the runner already supports—such as updating a downstream URL, mapping, step order, or timeout—an operator can edit the workflow in the UI and save it. The running bridge then uses the updated definition for new executions, often avoiding application code changes or a caller redeployment. This makes supported integration changes close to a no-code workflow; it does not cover new engine capabilities or new input data that callers do not already provide.

Workflow configuration becomes part of the integration contract, while callers keep their existing business payload shape where possible.

In practice, that means registering a definition once under a unique API name, then triggering it with the live request context. Independent steps can be grouped for parallel execution, and saved outputs can feed later steps. The service takes responsibility for the supported orchestration, while callers use a consistent REST or gRPC entry point.

---

What the Current Bridge Provides

---

The API Bridge has two cooperating processes: cmd/bridge exposes the gRPC service, and cmd/bridge-api provides a REST API and embedded web UI. The REST process is an adapter to the bridge, not a second workflow engine. Workflow definitions can be registered, cached, and persisted to a JSON file.

The runner executes top-level steps in declaration order. A parallel step runs its children concurrently, and saved outputs can be referenced by later steps. It supports per-step timeouts and returns the last step's HTTP response. It does not provide a general dependency-graph scheduler, automatic HTTP-status branching, step retries, compensation, or response merging. dependsOn-style DAG scheduling would be an extension to this execution model.

The bridge also provides live JSON↔XML conversion. Send the actual JSON or XML body to the /transform HTTP endpoint or pass raw payload bytes to the gRPC Transform method; no separate wrapper payload is needed just to request conversion. The HTTP endpoint can infer the input format from Content-Type, and X-Target-Format selects the target format.

---

An API-First Contract and Companion UI

---

Separating workflow management from execution gives operators and callers a focused set of operations:

Operation Purpose Register or update Save a workflow under a unique API name. Fetch or list Inspect one or all registered definitions. Delete Remove a registered definition. Execute Trigger a registered workflow by name; gRPC also supports inline configuration for one-off execution.

The companion UI uses the REST API to list, create, edit, delete, and test workflows. A test run can show the workflow result and step trace. Saved updates take effect in the running instance; production teams should review and version definitions and plan for access control and multi-instance consistency.

---

Architecture and Execution

---

For a registered workflow, the execution path is straightforward:

1. Look up the workflow by API name in the in-memory registry.
2. Process top-level steps in declaration order; execute children of a parallel step concurrently.
3. Resolve supported values from the live request or prior step outputs, then make the downstream HTTP calls.
4. Return the last step's response and publish an execution trace.

The JSON-file persistence and in-memory lookup avoid reading storage on every execution. Multi-instance deployments still need a strategy to share and coordinate workflow updates.

Request Sequence

Direct gRPC callers skip the REST adapter. The /transform endpoint is a separate path for live payload conversion, not a prerequisite for workflow execution.

A Small Workflow Example

A registered definition can take values from the live request, save one step's result, and use that result in the next step. The order of these entries is significant:

{
    "steps": [
        {
            "id": "getCustomer",
            "type": "http",
            "target": { "url": "https://api.example.com/customers/{customerId}", "method": "GET" },
            "input": { "params": { "customerId": "${context.customerId}" } },
            "output": { "saveas": "customer" },
            "timeout": "4s"
        },
        {
            "id": "getOrders",
            "type": "http",
            "target": { "url": "https://api.example.com/orders", "method": "GET" },
            "input": { "query": { "customerId": "$.customer.id" } },
            "output": { "saveas": "orders" },
            "timeout": "4s"
        }
    ]
}

Here, getCustomer runs first and its saved output is available to getOrders. ${context.customerId} resolves a matching key from the live query, headers, or top-level JSON body. A parallel step groups independent requests.

---

Interfaces and Latency

---

Independent calls can run in parallel. Ignoring engine overhead and assuming no queueing or retries, a parallel batch approaches the duration of its slowest child rather than the sum of child durations. Real latency still depends on network conditions, scheduling, downstream capacity, and service limits. Parallelism reduces waiting; it does not remove downstream bottlenecks.

Why REST and gRPC?

The REST adapter is convenient for browser-based tools, scripts, and HTTP clients. Service-to-service callers can use gRPC. Both interfaces lead to the same bridge workflow service, so workflow logic is not duplicated across transports. The gRPC server uses its configured JSON codec; a .proto file does not mean the wire format is protobuf-binary.

Concurrency and Caching

Workflow definitions are cached in memory so executions do not need to read the persistence file on every request. This reduces orchestration overhead, but downstream response time, network conditions, and service capacity remain the dominant factors in many real workflows.

---

Example: Customer Data Enrichment

---

Suppose a client needs a customer profile, recent orders, and loyalty information. A registered workflow can fetch the customer first, use the saved customer ID in a later request, and then run independent enrichment calls in a parallel step. The caller makes one request instead of coordinating each service itself.

For example, getCustomer can save its response as customer; getOrders can then use $.customer.id; independent loyalty or recommendation calls can run together. The workflow is registered once, and each execution supplies request values rather than resending the definition.

The result returned is the last step's response. If a client needs a new document combining several results, configure a downstream aggregation call or add response composition.

This pattern applies to composite endpoints generally: send the original request context once, reuse saved output in later steps, and group independent work where useful.

---

Why Not Use a Full Workflow Platform?

---

This bridge is designed for lightweight API orchestration rather than durable business-process execution.

Platforms such as Temporal, Camunda, and Netflix Conductor provide capabilities such as:

• Durable execution
• State persistence
• Automatic retries
• Compensation workflows
• Long-running processes
• Human approval tasks

The bridge intentionally focuses on:

• Request-time API composition
• Declarative mappings
• Low operational overhead
• REST/gRPC integration
• Configuration-driven orchestration

If your workflow must survive process restarts, run for hours or days, or coordinate compensating transactions, a dedicated workflow platform may be a better choice.

---

Reuse Across Teams

---

The same registered workflow can serve mobile, web, partner integrations, or internal tools through the supported request interfaces. A team can make supported mapping or timeout changes centrally rather than coordinating a client release for each adjustment. This shifts a meaningful portion of composition from application code into configuration, while complex policy and business logic still require engineering work.

---

Illustrative Extension: Loan Origination

---

A loan application may require identity checks, account creation, bank validation, and disbursement. Aadhaar and PAN checks could run in parallel, followed by later steps in declaration order. The sketch below is not a production-ready loan workflow: the identity decision still needs an explicit validation gate before account creation.

The JSON sketch shows declaration-order steps and parallel KYC calls. There are no dependsOn fields: top-level order establishes the sequence. The runner does not branch on an identity mismatch, so the validation gate must be implemented before using this flow for account creation.

{
    "steps": [
        {
            "id": "kycChecks",
            "type": "parallel",
            "failurePolicy": "fail",
            "children": [
                {
                    "id": "validateAadhaar",
                    "type": "http",
                    "target": { "url": "https://kyc.example.com/aadhaar/verify", "method": "POST" },
                    "input": { "body": { "aadhaarNumber": "${context.aadhaarNumber}", "name": "${context.applicantName}" } },
                    "output": { "saveas": "aadhaarResult" },
                    "timeout": "4s"
                },
                {
                    "id": "validatePan",
                    "type": "http",
                    "target": { "url": "https://kyc.example.com/pan/verify", "method": "POST" },
                    "input": { "body": { "panNumber": "${context.panNumber}", "name": "${context.applicantName}" } },
                    "output": { "saveas": "panResult" },
                    "timeout": "4s"
                }
            ]
        },
        {
            "id": "validateIdentity",
            "type": "http",
            "target": { "url": "https://identity.example.com/cross-check", "method": "POST" },
            "input": { "body": {
                "aadhaarName": "$.aadhaarResult.name",
                "panName": "$.panResult.name",
                "applicantName": "${context.applicantName}"
            } },
            "output": { "saveas": "identityCheck" }
        },
        {
            "id": "createLoanAccount",
            "type": "http",
            "target": { "url": "https://loans.example.com/accounts", "method": "POST" },
            "input": { "body": {
                "applicantName": "${context.applicantName}",
                "identityRef": "$.identityCheck.referenceId",
                "loanAmount": "${context.loanAmount}"
            } },
            "output": { "saveas": "loanAccount" }
        },
        {
            "id": "validateBankAccount",
            "type": "http",
            "target": { "url": "https://banking.example.com/accounts/verify", "method": "POST" },
            "input": { "body": {
                "accountNumber": "${context.bankAccountNumber}",
                "ifsc": "${context.bankIfsc}",
                "loanAccountId": "$.loanAccount.id"
            } },
            "output": { "saveas": "bankValidation" }
        },
        {
            "id": "disburseLoan",
            "type": "http",
            "target": { "url": "https://loans.example.com/disbursements", "method": "POST" },
            "input": { "body": {
                "loanAccountId": "$.loanAccount.id",
                "bankAccountRef": "$.bankValidation.referenceId",
                "amount": "${context.loanAmount}"
            } }
        }
    ]
}

---

Asynchronous Inputs as a Custom Adapter

---

Not every workflow needs to begin with a synchronous call. A separate consumer service could read RabbitMQ, Kafka, or another asynchronous source, map an event into a bridge request, and invoke the workflow through REST or gRPC. Teams can customize that adapter for their message schema, routing, acknowledgements, retries, and dead-letter policies, and deploy it alongside existing services.

This is an extension pattern, not a built-in feature: RabbitMQ publishes outbound trace events, but workflow consumers for RabbitMQ or Kafka are not included. Keeping a consumer separate also lets teams scale and operate it independently.

---

Production Trade-offs

---

Orchestration centralizes workflow logic, but also makes the engine an important dependency. Plan for its availability, capacity, monitoring, and failure handling. Earlier steps may have completed before a later step fails, leaving partial side effects. A timeout also does not prove that an operation failed: the downstream service may have completed the action but lost its response. For operations such as account creation or disbursement, retries need idempotency keys or downstream deduplication to prevent duplicate effects.

Treat workflow definitions as production artifacts. Version and review them so teams can reproduce runs and roll back safely; changing a definition in place can alter behavior for new requests. Multi-instance deployments need shared persistence or another coordination strategy to keep definitions consistent.

Protect workflow registration and downstream credentials with suitable access controls and secret management. Use TLS or mTLS where appropriate. Traces may contain sensitive request or response data, so minimize logged payloads, mask personal information, and restrict trace access and retention.

---

Summary

---

Declarative orchestration can move supported integration changes out of client code and into centrally managed workflows. The API Bridge provides workflow registration and execution, REST and gRPC entry points, and live JSON↔XML conversion. Its runner processes steps in declaration order, supports parallel children and per-step timeouts, and returns the final step's response.

The broader goal is to let clients keep sending familiar data through simple REST/gRPC calls, with separately customizable adapters for asynchronous inputs. That expanded experience—including broker consumers and richer workflow controls—is work in progress, not a claim about features already shipped.