window.ai.contextCache

[hemanth]

Use Case

The window.ai.contextCache API allows web applications to efficiently manage and reuse context information for AI models. This is particularly useful for applications that involve ongoing conversations or require maintaining state across multiple AI interactions, such as chatbots, virtual assistants, or context-aware content generation tools.

By caching context, applications can:

1. Improve response relevance in multi-turn conversations
2. Reduce latency by avoiding the need to resend full conversation history
3. Optimize resource usage by managing context size

API Description

interface ContextCacheOptions {
  maxSize?: number; // Maximum number of tokens or characters to store
  ttl?: number; // Time-to-live in milliseconds
}

interface ContextEntry {
  id: string;
  content: string;
  timestamp: number;
}

interface WindowAI {
  contextCache: {
    add(id: string, content: string): Promise<void>;
    get(id: string): Promise<string | null>;
    update(id: string, content: string): Promise<void>;
    delete(id: string): Promise<void>;
    clear(): Promise<void>;
    setOptions(options: ContextCacheOptions): Promise<void>;
  };
}

interface Window {
  ai: WindowAI;
}

Methods

• add(id: string, content: string): Adds a new context entry to the cache.
• get(id: string): Retrieves a context entry by its ID.
• update(id: string, content: string): Updates an existing context entry.
• delete(id: string): Removes a context entry from the cache.
• clear(): Removes all entries from the cache.
• setOptions(options: ContextCacheOptions): Configures cache behavior.

Example Usage

async function manageConversationContext(conversationId, newMessage) {
  // Configure cache
  await window.ai.contextCache.setOptions({ maxSize: 1000, ttl: 3600000 });

  // Retrieve existing context
  let context = await window.ai.contextCache.get(conversationId);

  // Update context with new message
  context = (context ? context + "\n" : "") + newMessage;
  await window.ai.contextCache.update(conversationId, context);

  // Use updated context in AI interaction
  const response = await someAIFunction(context);

  return response;
}

This API provides a simple yet flexible way to manage context information for AI interactions in web applications.