The Dynamic Discovery Workflow#

When an agentic system connects to an MCP-based server, dynamic tool discovery operates through a standard client-server request-response lifecycle:

1. Discovery & Handshake (tools/list): Upon startup or user selection, the AI client sends a standardized RPC request (such as tools/list) to the connected MCP server or gateway. The server returns a structured array of all available capabilities, complete with exact parameter requirements and JSON schemas.
2. In-Context Prompt Synthesis: The AI agent converts these JSON schemas into system-level instruction definitions. Because the protocol uses standardized formats (like JSON Schema for arguments), the model knows precisely what inputs are required (e.g., string format, required parameters, default values) without developer intervention.
3. Intent Matching & Planning: When a user submits a prompt (e.g., “Export this table to Google Sheets”), the model analyzes the dynamic tool definitions currently loaded in its context window. If a registered tool matches the intent, the model generates a structured JSON payload containing the arguments.
4. Invocation & Output Binding (tools/call): The client receives the model’s structural tool call and sends a tools/call RPC message to the target MCP server with the generated arguments. The MCP server executes the underlying code/API call and returns a standardized result payload back to the agent context.

MCP Market is NOT “MCP Server/Hub”

Although matching syntatically, MCP Market and MCP Server/Hub represent two different layers of the ecosystem:

• MCP Market is a Public Community Directory/Registry. It serves as a human-facing web catalog (like npm, PyPI, or an app store listing) where developers publish, browse, and discover open-source MCP server implementations, installation guides, and documentation. It does not sit inside our active application request lifecycle or execute code for our AI agent.
• MCP Server/Hub is an Active Runtime Gateway / Backend Server. It is the live infrastructure endpoint (such as our backend Node.js server or a hosted container) that directly accepts streamable HTTP/SSE requests, executes the underlying tool logic, handles token authorization, and communicates back and forth with the LLM at runtime via JSON-RPC protocol messages.

1 - Unified Transport & Protocol Engine (Data Plane)#

This component handles low-level network connectivity, protocol parsing, and response framing:

• JSON-RPC Dispatcher: Deserializes incoming tools/list and tools/call RPC payloads and routes them to the correct execution handler.
• Streamable Network Adapters: Exposes stateful, bidirectional streaming transports—such as Server-Sent Events (SSE), WebSockets, or Streamable HTTP - over TLS to support continuous agent execution loops.

2 - Semantic Discovery & Context Trimming Engine#

Exposing hundreds of tool schemas at once exhausts the LLM’s context window and skyrockets token costs.

• Vector Tool Index: Stores embeddings of tool descriptions and parameter schemas.
• Dynamic Search & Filtering: When an agent queries the hub, this engine performs semantic similarity matching over the prompt to retrieve and expose only the top $N$ relevant tool schemas during the tools/list handshake.

The embeddings are generated from tool metadata documents during an automated indexing pipeline, and the vector search is performed against a centralized Vector Database (Registry).

3 - Protocol Adapter & Translation Layer#

When building an AI MCP server with hundreds of tools, we face a major issue: our AI agent speaks only MCP (Model Context Protocol / JSON-RPC), but the real-world software services we want to trigger (like Spotify, Salesforce, or Google Calendar) do not speak MCP—they speak standard REST, GraphQL, or gRPC.

Instead of writing custom backend code (like spotify.ts or salesforce.ts) to bridge every single service by hand, the Protocol Adapter automates this process.

4 - Identity, Authorization & Token Exchange (Control Plane)#

AI agents must execute actions using the specific end-user’s permissions rather than over-privileged system credentials.

• Identity Context Propagation: Intercepts incoming tools/call requests and extracts the user’s bearer token (OAuth2/OIDC).
• Token Vault Integration: Injects user-specific OAuth refresh/access tokens into downstream requests to ensure zero-trust, role-based access control (RBAC)

5 - Governance, Safety & Telemetry.#

In an autonomous environment, this layer prevents malicious agent behavior and system abuse.

• Prompt Injection & Tool Poisoning Guardrails: Inspects incoming inputs and returning execution results for adversarial payload patterns before returning them to the LLM.
• Rate Limits & Circuit Breakers: Enforces strict execution budget caps, execution timeouts, and rate limits to stop runaway recursive agent loops.
• Audit Logging: Centralizes telemetry, tracking token usage, latency, user context, and exact input/output payloads.

Authentication Flow (OAuth 2.0 with PKCE) for Single Integration (Google Calendar)#

1
// Updating tool execution to pass user credentials dynamically
2
async fetchToolSchemas(): Promise<MCPToolSchema[]> {
3
  const response = await this.client.listTools();
4

5
  return response.tools.map((tool) => ({
6
    name: tool.name,
7
    description: tool.description || "",
8
    inputSchema: tool.inputSchema as any,
9
    execute: async (args: any, userContext: { userId: string; userToken?: string }) => {
10

11
      // Inject user's specific OAuth Access Token into the MCP call
12
      const result = await this.client.callTool({
13
        name: tool.name,
14
        arguments: args,
15
        // Pass credentials in metadata/headers if supported by the MCP server,
16
        // or ensure the sub-process environment handles multi-tenant auth
17
        _meta: {
18
          authToken: userContext.userToken
19
        }
20
      });
21
      return result;
22
    }
23
  }));
24
}

Authentication Flow (OAuth 2.0 with PKCE) for Multiple Integrations#

To handle different authorization mechanisms gracefully across third-party MCP tools, we need an Adapter Pattern for Authentication Strategy in our backend.

Every service (Google Calendar, Spotify, Notion, YouTube Music) uses a different credential type:

• OAuth 2.0 PKCE (Google Calendar, Spotify) $\rightarrow$ Requires refresh_token and access_token rotation.
• API Key / Secret (OpenWeather, Notion) $\rightarrow$ Requires a static user-provided string.
• Cookie / Session Token (YouTube Music) $\rightarrow$ Requires raw auth cookies.

Instead of writing custom backend auth logic for each integration, the system can choose to model authentication uniformly across frontend, Database, and MCP Server Wrappers.

1
export interface UserIntegration {
2
  id: string;
3
  userId: string;
4
  provider: string; // "google-calendar", "youtube-music", "notion"
5
  authType: "oauth2" | "api_key" | "session_cookie";
6

7
  // Encrypted JSON string holding whatever credentials that provider needs
8
  encryptedCredentials: {
9
    accessToken?: string;
10
    refreshToken?: string;
11
    apiKey?: string;
12
    expiresAt?: number;
13
    rawHeader?: string;
14
  };
15
}

1
export interface AuthStrategy {
2
  // Returns clean credentials ready for the MCP server process
3
  getValidCredentials(userId: string): Promise<Record<string, string>>;
4
}
5

6
// auth/GoogleOAuthStrategy.ts
7
export class GoogleOAuthStrategy implements AuthStrategy {
8
  async getValidCredentials(userId: string): Promise<Record<string, string>> {
9
    const integration = await db.findIntegration(userId, "google-calendar");
10

11
    // Check if access_token is expired; refresh if necessary
12
    if (Date.now() >= integration.encryptedCredentials.expiresAt) {
13
      const newTokens = await refreshGoogleToken(integration.encryptedCredentials.refreshToken);
14
      await db.updateIntegration(integration.id, newTokens);
15
      return { authorization: `Bearer ${newTokens.accessToken}` };
16
    }
17

18
    return { authorization: `Bearer ${integration.encryptedCredentials.accessToken}` };
19
  }
20
}
21

22
// auth/ApiKeyStrategy.ts
23
export class ApiKeyStrategy implements AuthStrategy {
24
  async getValidCredentials(userId: string): Promise<Record<string, string>> {
25
    const integration = await db.findIntegration(userId, "notion");
26
    return { "x-api-key": integration.encryptedCredentials.apiKey! };
27
  }
28
}

1
export class ExternalMCPServerWrapper {
2
  private config: MCPServerConfig;
3
  private authStrategy: AuthStrategy;
4

5
  constructor(config: MCPServerConfig, authStrategy: AuthStrategy) {
6
    this.config = config;
7
    this.authStrategy = authStrategy;
8
  }
9

10
  async executeTool(toolName: string, args: any, userId: string) {
11
    // 1. Fetch valid user credentials dynamically (refreshes OAuth if expired)
12
    const authHeaders = await this.authStrategy.getValidCredentials(userId);
13

14
    // 2. Dispatch via stdio/SSE using the formatted credentials
15
    return await this.client.callTool({
16
      name: toolName,
17
      arguments: {
18
        ...args,
19
        // Option A: Pass credentials inside _meta if using dynamic runtime auth
20
        _meta: { authHeaders }
21
      }
22
    });
23
  }
24
}