A secure, extensible AI chat platform

Most Enterprise AI chat initiatives don't last because the architecture gets built for one use case, and every new one means rebuilding identity, audit logging, and deployment pipelines from scratch. With AI adoption accelerating across business units, our client needed a platform that survives the proof of concept and scales with the business.

Here's how we built a governed agent platform where extensibility is structural.The plugin model handles the full range of what enterprise users need in a single conversation with answers from documents, live data from backend systems, and actions triggered against them — without any of it hardcoded into the agent.

Defining success early

We resisted the instinct to build for the immediate need only. We created a stable core with every new capability pluggable from day one.

Knowledge, not just retrieval

Answer from stable sources across policies, FAQs, documentation and fetch live, user-specific data from backend systems all within the same conversation.

Production-grade from day one

Secure, auditable and operable without a dedicated ops team with governance built in from the onset

Extensible without touching the core

New use cases add capability to the platform without rebuilding it. A stable core with every capability pluggable from day one.

A clear contract for tech teams

Developers extend through a consistent plugin interface. Administrators control what ships and when. No hardcoded capabilities means no full redeploy.

Extend without rebuild

The architecture separates and resolves three concerns that are almost always tangled - runtime, management, and plugins. Every section of this platform was designed to eliminate a specific reason enterprise AI projects get abandoned

A stateless runtime
The runtime is a conversational agent built on Azure OpenAI. For each user turn it decides whether to retrieve from a knowledge base, call a live system, or both — with no hardcoded logic about which capabilities exist. That knowledge lives in the plugin registry, keeping the core engine clean and stable.

Extensibility with different plugins
RAG for document answers, Tool for live API calls, Hybrid for retrieve-then-act flows, and Prompt for scenario-specific behavior. The management API governs the lifecycle — draft, publish, disable — without touching the runtime. Changes reach production via Storage Queues, no redeploy required.

Security and visibility
The admin portal gives non-technical administrators full visibility across plugins, access controls and audit logs. Role-based access is enforced through Entra ID and Managed Identity. Telemetry, alerting and secrets management are unified across every layer

The tech stack

Staying within Azure was a deliberate choice. The client's existing investment in Microsoft tooling across Active Directory, Azure tenancy and compliance frameworks made it the right foundation for governance, not just compute.

The stack maps cleanly to the architecture: Azure OpenAI for the model with the enterprise security controls that come built into the Azure estate, AI Search for hybrid retrieval at scale and Functions to provide the serverless execution model for plugins.

The result

An AI platform the business can keep building on

Faster time to capability, reduced cost

Reduction in time to deploy a new AI capability from weeks to days

No deployment risk to the running platform

Governance built into the architecture

Role-based access controls and full audit logging

Zero-secret architecture and ready for compliance review from day one

Governance is built in to the architecture

New business units adopt AI chat with ease

Reusable across varied department workflows

Every new team adds plugins to a platform that is already governed and monitored

Clean engineering, reduced complexity

Clear separation between runtime, management plane, and plugin layer

Platform gains capability over time without accumulating complexity

A living AI Chat platform

Building with AI is straightforward. Building Agentic AI an enterprise can govern, extend, and operate without rebuilding is the real value creation.

Near-term improvements already in progress include maturing compute consolidation across worker functions, deepening RAG quality through evaluation loops and citation accuracy improvements, and extending the admin portal with richer observability dashboards. Each improvement lands without disrupting what is already running as extensibility is an architecture decision, not an afterthought.

Beyond the model, it is the architecture around it that makes this a living platform.

Let’s talk about your project

Reach out for a non-committal chat

Alexander Kongshaug