Architecture

Backend databases often reside on infrastructure separate from cloud environments — legacy on-premises systems, managed database services, or security-isolated namespaces. Exposing these databases to public networks creates security vulnerabilities. Traditional VPN configurations add operational complexity and require firewall rule management across teams.

Red Hat Service Interconnect solves this by establishing encrypted, application-level connections between OpenShift and remote databases. No VPNs. No exposed ports. The solution creates a local Kubernetes service proxy that transparently forwards traffic to the remote database.

Service Interconnect uses a set of resources to establish and manage the connection. Here’s how it looks:

Once configured, Apache Camel integrations connect to the database as if it were local — standard PostgreSQL connection strings, no special network configuration. The Service Interconnect layer handles routing, encryption, and access control beneath the application.

This pattern extends to multi-site deployments spanning clusters, regions, and cloud providers. Sites discover each other through credentials (AccessGrant/AccessToken), creating secure meshes that persist across all exposed services.

A typical data ingestion flow follows this pattern: the platform exposes an API endpoint via Apache Camel to receive incoming data. The integration transforms data as needed, stores it in backend databases via Service Interconnect, and generates documents that are uploaded to object storage provided by OpenShift Data Foundation. All data movement happens over encrypted connections without direct database exposure.

In the demonstration scenario, this pattern is applied to order processing: EDI purchase orders arrive at the Camel endpoint, are transformed to XML format, stored in a database, and rendered as PDF invoices uploaded to S3 storage.

Connecting messaging platforms directly to each other creates tight coupling. When two platforms depend on each other’s APIs, adding a third platform requires modifying both existing integrations. The number of point-to-point connections grows exponentially as the platform expands.

The architecture decouples messaging platforms by introducing Apache Kafka as a streaming backbone. Neither platform requires direct awareness of the other — they publish and consume messages through common Kafka topics. This decoupling enables independent scaling and simplifies extensibility.

To make this pattern work cleanly, the architecture adopts a standard data model as the common language on Kafka. Instead of translating directly between platform-specific formats, each integration transforms in two steps:

This means every system integrates with one common interface, not with every other system individually. Adding a new messaging platform requires implementing only two flows: platform → Kafka (inbound) and Kafka → platform (outbound). The existing infrastructure and other integrations remain untouched.

In the demonstration scenario, bidirectional messaging between Matrix (customer-facing) and Rocket.Chat (support agents) illustrates this pattern with four distinct flows:

Each flow is an independent Apache Camel integration. Kafka partitions enable parallel processing. Message ordering is preserved within partitions while allowing horizontal scaling across consumer groups.

Support agents resolving customer issues typically query databases manually, update records across systems, and regenerate documents. The process is time-consuming and error-prone. Scaling support operations requires hiring proportionally more agents.

The platform enables support agents to interact with an AI assistant using natural language directly within their messaging interface. Powered by Agentic Apache Camel and Model-as-a-Service (MaaS), the agent interprets requests, autonomously selects tools, and executes backend operations.

The AI agent follows a reasoning loop — it consults the LLM via MaaS, which decides what actions to take. Camel then executes the appropriate tools via MCP (update the database, regenerate a PDF invoice in S3), feeds the results back to the LLM, and returns the final answer to the support agent in Rocket.Chat.

The diagram illustrates the agentic reasoning loop:

① The support agent sends a natural language request via Rocket.Chat

② Rocket.Chat forwards the message to the Agentic Camel integration

③ The Agentic Integration reasons with the AI Model (MaaS) to determine the appropriate actions

④ Based on the reasoning, Camel executes MCP tools which are connected to backend systems (Database, S3/ODF, HelpDesk)

⑤ Results flow back to the Agentic Integration

⑥ The final synthesized response returns to the support agent

The diagram below shows how the Agentic Integration (once MCP is active) routes tool calls from the AI model. Two MCP servers provide distinct toolsets — Node.js and Camel Quarkus. When the AI model needs a tool, the Agentic Integration routes the request to the appropriate MCP server, which executes the tool against the backend systems and returns the result.

In the demonstration scenario, a support agent might ask "Fix invoice 1025 shipping address to 123 Main St and regenerate the PDF." The LLM recognizes this requires three operations: query the invoice, update the shipping address, regenerate the PDF. Camel executes each operation, validates the update succeeded, renders the new PDF, uploads it to S3, and reports completion back to the agent.

Once AI agents operate in production, organizations need controls. Who can access the LLM endpoint? How many tokens can each service tier consume? Without governance, uncontrolled API usage can exhaust budgets in hours.

The architecture applies a governance layer in front of LLM endpoints using Red Hat Connectivity Link. Built on the Kubernetes Gateway API standard and the Kuadrant project, Connectivity Link enforces infrastructure-level policies without requiring application code changes.

Two policy types control AI access:

AuthPolicy manages identity and authorization. API keys identify callers and assign them to groups (for example, service tiers such as Gold and Silver). The policy validates keys, extracts group membership, and passes that context downstream for rate limiting decisions.

TokenRateLimitPolicy enforces consumption limits based on actual token usage, not request counts. Each service tier gets its own token budget. Organizations can define fallback strategies when higher-cost tiers exhaust their quotas — for instance, falling back from Gold tier to Silver tier with reduced analysis depth, or deferring requests when all tiers are exhausted.

This token-aware rate limiting is critical for AI workloads. Traditional request-count limits don’t account for the fact that one request can consume 100 tokens while another consumes 10,000. Token-based policies measure actual consumption, enabling fair allocation across tiers.

Policies attach directly to HTTPRoute resources in the Gateway API model:

Application code sees none of this. The AI agent makes requests to the LLM endpoint. Connectivity Link intercepts traffic at the gateway, enforces authentication, tracks token consumption, and applies rate limits before forwarding requests to the LLM service.

The end-to-end request flow shows how the support agent’s interaction flows through the governance layer:

Policies are defined as Kubernetes Custom Resources, applied with oc apply, stored in Git, and reviewed through standard change management processes. No separate management console. No vendor control plane. Governance follows the same operational patterns as application deployment.

The architecture is designed for horizontal scaling across all components:

Apache Camel integrations scale by increasing replica counts. Each instance subscribes to the same Kafka consumer group. Kafka automatically load-balances partitions across available consumers. Adding replicas increases throughput without configuration changes.

Kafka scales through partitioning. Topics are divided into partitions distributed across brokers. Consumers process partitions in parallel. Producers distribute messages across partitions using configurable strategies (round-robin, key-based hashing, custom logic).

Service Interconnect supports multiple Sites in a mesh topology. Sites can be added dynamically without disrupting existing connections. Traffic routes across the mesh automatically.

Connectivity Link gateways handle high request throughput. Rate limiting state is distributed across gateway instances. Token budgets are tracked consistently across the deployment.

OpenShift Data Foundation provides distributed object storage that scales independently of compute resources. S3-compatible endpoints support parallel uploads and downloads.

Under increasing traffic volumes, each component scales independently. If outbound traffic to support agents increases, scale the Kafka → Rocket.Chat Camel integration without touching Matrix integrations or the AI agent.

The decoupled architecture makes adding new systems straightforward. Once the standard data model is defined, integrating new platforms follows a consistent pattern.

Adding a new messaging platform: Implement two Camel integrations — one transforms from the platform’s native format to the standard model and publishes to Kafka, the other consumes from Kafka and transforms to the platform’s format. No changes to existing integrations. No modifications to Kafka topics. The new platform participates in all existing message flows automatically.

Adding a new AI tool: Define the tool’s signature and implement it as a Camel route. Register the tool with the Agentic Camel configuration. The LLM can now invoke it autonomously during reasoning loops. Existing tools remain unaffected.

Adding a new service tier: Define a new HTTPRoute with distinct rate limit policies. Assign API keys to the new tier’s group. Token budgets are enforced independently from existing tiers.

This extensibility comes from architectural choices made early in the design:

The pattern supports both horizontal expansion (more messaging platforms, more tools, more tiers) and vertical enhancement (richer analysis, deeper integrations, additional governance controls).