Octave Launches Secure Multi-Tenant AI Foundation on Amazon Bedrock

Summary:Octave has launched a secure multi-tenant AI foundation on Amazon Bedrock, leveraging Cohere Embed v4 and Amazon Bedrock AgentCore to ensure data isolation and secure agent execution. The architecture supports embedding and retrieval of 2 billion documents and has been successfully applied in a Complaint Triage Assistant, significantly improving compliance efficiency while reducing cross-tenant data leakage risks.

---

Technical Architecture and Core Mechanisms

Octave has developed a secure multi-tenant AI foundation to address critical challenges in handling sensitive data within highly regulated industries. The core components of this architecture include:

1. Data Preparation Layer: Utilizing Cohere Embed v4 on Amazon Bedrock for document embedding, transforming over 2 billion documents into vector representations and storing them in per-tenant vector stores on Amazon OpenSearch Service.

   • Technical Highlights: Embed v4 supports multi-modal data (text and images) embedding, eliminating the need for separate processing of different data types. Additionally, native multi-language support reduces the need for translation preprocessing.
   • Trade-offs and Engineering Costs: Although Embed v4 offers powerful features, its high-dimensional (256-1536) vector representations impose higher storage and computational resource requirements. Through testing, Octave opted for a 512-dimensional vector representation, which maintains accuracy while reducing storage needs by approximately 66%.

2. Secure Execution Layer: Leveraging Amazon Bedrock AgentCore to ensure secure agent execution. Each user session runs in a dedicated microVM, isolating CPU, memory, and filesystem, ensuring complete data isolation between different tenants.

   • Technical Mechanisms: AgentCore Gateway authenticates and authorizes user requests, ensuring that agents can only access authorized data and tools.
   • Trade-offs and Engineering Costs: Hardware-level isolation provides higher security but also increases resource overhead. Octave mitigated the performance impact by optimizing microVM startup and teardown times.

3. Content Protection Layer: Applying Amazon Bedrock Guardrails at both data input and output stages to filter out malicious content and prevent sensitive information leakage.

   • Technical Mechanisms: Guardrails filter at three key points: attachment extraction, retrieval results, and synthesized output. Additionally, AgentCore Code Interpreter runs in a sandbox environment, preventing network access and persistent storage.

Engineering Challenges and Solutions

1. Cross-Tenant Data Isolation: To prevent cross-tenant data leakage, Octave pushed the isolation mechanism from the application layer into the runtime layer, using microVM and AgentCore to achieve hardware-level isolation.
2. Handling Untrusted Content: Facing untrusted content from external sources, Octave adopted a defense-in-depth strategy, combining input filtering, output filtering, and sandboxed execution to ensure malicious content cannot affect the system.
3. Large-Scale Document Embedding: To complete the embedding of the largest tenant within a day, Octave optimized the embedding pipeline, using batch processing and parallel computing to increase throughput to 280,000 tokens per second.

Real-World Application and Results

Octave's Complaint Triage Assistant has been in production for three months and has significantly improved compliance efficiency: a complaint that previously required starting from scratch now comes with relevant history pre-assembled.

• Efficiency Gains: The processing time per investigation has been reduced from 4-6 hours to just the time needed to process new information.
• Shorter Development Cycles: The development cycle for new AI features has been reduced from months to weeks, with the shared data preparation, secure execution, and retrieval layers reducing deployment time for new use cases by 50%.

Future Outlook

Octave plans to extend this foundation across the Reliance suite, including audit readiness monitoring, incident narrative generation, and predictive quality risk scoring.

Developer Recommendations

For platforms facing similar challenges, Octave's experience shows that building a secure multi-tenant AI foundation can effectively resolve the tension between data isolation and AI feature development. Consider the following:

• Choose the Right Embedding Model: Select an embedding model that supports multi-modal and multi-language data based on your data characteristics, and conduct thorough testing to find the optimal dimension.
• Implement Hardware-Level Isolation: Use technologies like microVM to achieve data isolation between tenants, ensuring security.
• Optimize the Embedding Pipeline: Increase embedding efficiency through batch processing and parallel computing to meet large-scale data processing needs.
• Apply Content Protection Mechanisms: Implement filtering mechanisms at data input and output stages to prevent malicious content injection and sensitive information leakage.

---

Source: AWS Machine Learning Blog (2026-10-20)

— END —

Tags: #Octave Follow #Amazon Bedrock Follow #AI Foundation Follow #Multi-Tenant Follow #Security Follow