MosaicLeaks Exposes How AI Research Agents Leak Private Enterprise Data

By VEONIB | 2026-07-12

Quick Answer

MosaicLeaks is a new privacy benchmark from ServiceNow researchers that proves AI research agents frequently leak sensitive enterprise information through their web search queries, with leakage rates reaching 34% before mitigation and dropping to under 10% after privacy-aware reinforcement learning training.

TL;DR

Table of Contents

According to MosaicLeaks: Can your research agent keep a secret? published by ServiceNow on the Hugging Face blog on June 18, 2026, AI research agents that combine private local documents with public web search create a dangerous privacy vulnerability known as the mosaic effect. The researchers built a benchmark of 1,001 multi-hop research chains where agents must interleave local and web queries, and discovered that leakage rates remained alarmingly high across all tested models. Simple prompting failed to stop leaks, and performance-focused training actually made the problem worse. For ecommerce businesses using AI agents to analyze proprietary product data, customer information, or internal strategy documents alongside public web information, this research carries urgent practical implications. This article analyzes MosaicLeaks from VEONIB's perspective, exploring how AI-powered video generation workflows, ecommerce analytics, and enterprise agent deployments must fundamentally redesign their approach to privacy.

Hero Image Alt Text: MosaicLeaks privacy benchmark diagram showing how AI agent web queries combine to reveal private enterprise information through the mosaic effect Caption: ServiceNow's MosaicLeaks demonstrates how individual benign queries can collectively expose sensitive business data OG Image Title: MosaicLeaks Privacy Benchmark - AI Agent Leakage Analysis for Enterprise Suggested Visual: A visual showing scattered puzzle pieces labeled with individual web queries coming together to form a confidential document marked "PRIVATE"

The Privacy Risk Hidden in AI Research Agent Queries

Deep research agents—AI systems that can browse the web, read documents, and synthesize information—are increasingly deployed in enterprise environments. These agents access private local documents while simultaneously querying public web sources to answer complex multi-hop questions. The privacy risk stems from a fundamental architectural design: every web search an agent performs leaves a trace in external systems.

Original Fact: ServiceNow researchers designed MosaicLeaks to treat web queries as the primary leakage channel. The adversary never sees private documents or the agent's internal reasoning, only the cumulative query log. The goal is to determine whether an observer can infer private enterprise information from the pattern of queries alone.

The mosaic effect gets its name from the way individual pieces of information, harmless on their own, combine to reveal a complete picture. A research agent at a healthcare firm might search for "cloud migration 70% complete" alongside "January 2024 security disclosure" and "vendor breach response timeline." Each query appears routine. Together, they tell an observer that MediConn migrated 70% of its infrastructure by January 2025 and suffered a related security incident.

VEONIB Insight: For ecommerce businesses deploying AI agents, this risk is not theoretical. An agent analyzing internal sales data while researching competitor pricing or market trends could leak proprietary revenue figures, product launch dates, or customer demographics through its search queries. AI video generation workflows that combine internal product performance data with public trend analysis face the same vulnerability. Any system that touches both private and public information must treat query logs as a potential exposure vector. The mosaic effect means security teams cannot rely on reviewing individual queries for sensitivity—they must analyze the aggregate pattern.

Understanding MosaicLeaks: A New Privacy Benchmark for Enterprise AI

MosaicLeaks comprises 1,001 multi-hop research chains built over local enterprise documents and a controlled web corpus. The benchmark's key innovation is creating tasks that deliberately induce privacy leakage from enterprise documents while remaining solvable without leaking.

Original Fact: Each chain interleaves local and web sub-questions. The answer to one sub-question becomes a bridge entity in the next, forcing the agent to retrieve local information before forming the next useful web query. Local documents come from DRBench-style enterprise tasks, and web documents come from BrowseComp-Plus. The final dataset split contains 559 training chains, 98 validation chains, and 344 held-out-company test chains.

The construction process involves three stages:

Stage Action Purpose
Seed private facts Generate question-answer pairs from enterprise documents Covers internal metrics, dates, dollar amounts, named entities
Bridge documents Use previous answer to retrieve new document and generate next question Creates explicit local-web dependencies
Validate chains Check answerability, retrievability, source order, necessity of previous answer Ensures chains are solvable and leakage-prone

VEONIB Insight: The MosaicLeaks construction methodology reveals something crucial about real-world privacy risks: the most dangerous leaks happen when agents must connect private internal information with public external knowledge. In ecommerce, this maps directly to scenarios like analyzing proprietary conversion data while researching industry benchmarks, or evaluating internal inventory costs while comparing competitor pricing. The benchmark's deliberate design to create local-web dependencies mirrors exactly the workflows that ecommerce AI tools will increasingly automate. Businesses should audit their agent workflows to identify every point where private and public information must interleave.

Why Simple Privacy Prompts Fail to Stop Information Leakage

The most obvious mitigation strategy—simply telling the agent not to leak—proved ineffective in the MosaicLeaks experiments.

Original Fact: Researchers added a line to the Plan prompt instructing the agent not to issue web queries that leak local information. The results showed inconsistent effectiveness across models, with significant leakage remaining. For Qwen3-4B, the prompt lowered answer/full-information leakage from 34.0% to 25.5%, but strict chain success dropped from 48.7% to 44.5%. The primary behavioral change was fewer web queries, not consistently safer query construction.

The researchers noted that the privacy prompt caused agents to reduce their web search activity, which paradoxically decreased both leakage and task performance. Better models sometimes ignored the prompt entirely, while weaker models complied but sacrificed accuracy.

Model Baseline Leakage Prompted Leakage Baseline Accuracy Prompted Accuracy
Qwen3-4B 34.0% 25.5% 48.7% 44.5%
Other tested models Varies Inconsistent Varies Negative impact common

VEONIB Insight: This finding has direct implications for ecommerce AI deployments. Telling an AI agent "don't share sensitive information" is insufficient because agents lack a robust understanding of what constitutes sensitive information in context. A Shopify merchant's AI agent might not recognize that combining "70% inventory sold" with "Q3 2025 timeline" leaks a proprietary sales milestone. For AI video generation workflows that reference internal product performance data, simple privacy instructions are not reliable safeguards. Companies need architectural controls, not behavioral nudges. This also means that any platform offering "privacy-safe AI" based solely on prompt engineering is likely overpromising.

Training for Better Performance Made Privacy Leakage Worse

Perhaps the most counterintuitive finding in MosaicLeaks is that making agents better at their primary task—solving research chains—actually increased privacy leakage.

Original Fact: Researchers trained agents using reinforcement learning (RL), specifically a GRPO-based method, to optimize chain success rate. Strict chain success rose from 48.7% to 59.3%. However, answer/full-information leakage climbed from 34.0% to 42.8%. The agent learned to make more aggressive web queries to find answers faster, and those queries leaked more private information.

The behavioral analysis revealed that RL-trained agents issued more web queries per hop, searched more aggressively for bridge entities, and included more private context in their search terms. The desire to maximize task completion drove the agent toward leaking behavior.

Training Method Strict Chain Success Answer/Full-Information Leakage
No training (baseline) 48.7% 34.0%
RL for performance only 59.3% 42.8%
PA-DR (privacy-aware RL) 58.7% 9.9%

VEONIB Insight: This is a critical lesson for any business deploying AI agents. Standard optimization approaches that reward task completion will inadvertently reward privacy-violating behavior. For Amazon sellers using AI to analyze competitor listings while accessing their own sales data, or DTC brands running AI-driven market research, the "better" the agent becomes at answering questions, the more likely it is to expose proprietary information. The VEONIB perspective is clear: privacy metrics must be integrated into training objectives from day one, not added as an afterthought. Companies evaluating AI vendors should ask specifically how the training methodology addresses the privacy-accuracy tradeoff, and whether benchmark results account for leakage rates alongside task performance.

Privacy-Aware Deep Research: A Reinforcement Learning Solution

The MosaicLeaks team developed Privacy-Aware Deep Research (PA-DR), a reinforcement learning approach that jointly optimizes for chain accuracy and privacy preservation.

Original Fact: PA-DR uses a GRPO-based RL framework with a composite reward function that includes both chain success rewards and privacy penalties. The privacy penalty is calculated based on whether web queries contain private information from local documents. PA-DR raised strict chain success from 48.7% to 58.7% while reducing answer/full-information leakage from 34.0% to 9.9%.

The key innovation is "situational rewards"—the agent receives different privacy penalties depending on the context of each query. A query that includes private information when a public-only query would suffice incurs a larger penalty. The training also improved sample efficiency, requiring fewer training episodes to achieve both accuracy and privacy goals.

VEONIB Insight: PA-DR demonstrates that privacy and performance are not inherently opposed—they simply require deliberate architectural design. For ecommerce businesses, this means investing in AI systems that natively understand what information is public vs. private at the model architecture level, not through surface-level prompt engineering. AI video generation platforms like VEONIB that process product URLs containing proprietary data should consider similar architectural safeguards. The 10% leakage rate remaining after PA-DR, however, indicates that no solution is perfect. Businesses must implement layered defenses: architectural privacy controls, query log monitoring, and clear data classification policies.

Comparing Privacy Mitigation Approaches for Enterprise AI Agents

Approach Implementation Leakage Reduction Performance Impact Scalability
Privacy prompt only Single line in system prompt Inconsistent (5-15% reduction) Accuracy drops 4-10% High (easy to deploy)
RL for performance GRPO training on chain success None (leakage increased) Accuracy improves 10% Medium
PA-DR (proposed) GRPO with composite privacy rewards 71% reduction (34% to 9.9%) Accuracy improves 10% Medium-High
Query log auditing Manual or automated review of search logs Depends on vigilance No direct impact Low (human effort)
Data segregation Isolate private and public data sources High if perfectly implemented May limit agent capabilities Low (architectural redesign)

VEONIB Insight: The comparison reveals that PA-DR offers the best balance of leakage reduction and performance maintenance. However, ecommerce businesses should note that query log auditing remains essential as a complementary control. No automated system achieves 100% privacy protection. For Shopify merchants using AI to analyze proprietary sales data alongside public market trends, the recommended approach combines PA-DR-style privacy training with automated query monitoring and clear data handling policies.

Implications for AI Video Generation and Ecommerce Workflows

The MosaicLeaks findings have specific implications for AI-powered video generation in ecommerce contexts. Many AI video workflows already involve combining proprietary product data with public content.

VEONIB Insight: Consider a VEONIB workflow where a merchant inputs a product URL containing internal pricing, profit margins, and inventory levels. The AI generates product analysis, scripts, and video prompts that may reference these internal metrics. If the system accesses public competitor information or trending content to inform video strategy, the same mosaic effect risk applies. Web searches for "70% profit margin winter coats" combined with "Q4 2025 launch timeline" could leak proprietary business intelligence.

For the ecommerce use cases VEONIB serves—Shopify merchants, Amazon sellers, TikTok Shop sellers—the practical risks include:

The recommended countermeasures include:

The MosaicLeaks Architecture: Tool-Based Agent Harness

MosaicLeaks uses a simplified agent harness adapted from DRBench, providing the agent with four tools that mirror real-world enterprise agent deployments.

Original Fact: The agent can use Plan (produces local and web search queries), Choose (selects which retrieved documents to read), Read (attempts to answer the current hop from selected documents), and Resolve (decides whether to answer, read more, or plan another search). At each iteration, the model answers each sub-question with a short answer and justification, allowing individual hop evaluation with normalized string matching.

The tool-based architecture means each query is explicitly generated by the model and executed against either local or web document stores. This design choice makes the leakage channel visible and measurable, unlike end-to-end models where search behaviors may be internal.

VEONIB Insight: The tool-based architecture closely resembles how real ecommerce AI agents operate. A merchant's AI video generation system might use separate tools for "access product database" and "search trending content." The visibility of each tool call is both a risk and an opportunity. Businesses can monitor tool usage patterns for early warning signs of privacy leakage. For AI video workflows, the recommendation is to implement similar tool-level logging and analysis. The VEONIB platform philosophy aligns with this approach: every step from product analysis to video generation should be auditable and controllable.

Measuring Privacy Leakage Across Three Threat Levels

MosaicLeaks defines three distinct levels of privacy leakage, each representing a different threat scenario for enterprise deployments.

Original Fact: Intent leakage occurs when the adversary can infer the private research questions or goals the agent was trying to answer, based solely on the web-query log. Answer leakage means the adversary can answer private questions without seeing the private documents, using only the query log plus the question. Full-information leakage is the strongest case: the observer can discover and state verifiable true private facts without being told what to look for.

Leakage Level Adversary Knowledge Adversary Ability Business Impact
Intent leakage Query log only Infer research goals Competitors learn strategic focus areas
Answer leakage Query log + private questions Answer specific private questions Competitors answer proprietary questions
Full-information leakage Query log only Discover and state new private facts Uncontrolled exposure of business intelligence

VEONIB Insight: For ecommerce businesses, the three levels represent escalating risk scenarios. Intent leakage might reveal that a merchant is researching "Q4 holiday pricing strategies for luxury handbags," signaling strategic direction to competitors. Answer leakage could enable a competitor to determine "What was Brand X's Q3 profit margin on premium accessories?" Full-information leakage means an observer can discover entirely new private facts, such as "Brand X is launching a direct-to-consumer channel in Q2 2026." The PA-DR approach reduced answer/full-information leakage most effectively, which is the priority for most businesses. However, intent leakage may still be damaging—companies should assess their tolerance for each level based on competitive sensitivity.

Practical Business Risks for Enterprise AI Deployments

The MosaicLeaks research highlights several real-world scenarios where enterprise AI deployments face privacy exposure.

Original Fact: The researchers cite a healthcare firm example where an agent researching a cloud migration milestone referenced "70% infrastructure migration by January 2025" in web queries. Combined with public security disclosure dates and vendor names, an observer could reconstruct the private timeline.

For ecommerce businesses, analogous scenarios include:

VEONIB Insight: The most dangerous risk is not the obvious data dump but the slow, cumulative exposure of business intelligence through operational patterns. Ecommerce businesses deploying AI agents should conduct privacy impact assessments that specifically evaluate mosaic effect risks. The assessment should map every point where private and public data intersect in the agent's workflow, identify what information could leak through query patterns, and implement appropriate controls. For AI video generation, this means ensuring that internal performance data used for script creation does not inadvertently appear in web research queries related to content strategy.

Recommendations

For Shopify Merchants: Audit your AI tools to identify any workflow that combines private store data (sales, margins, inventory) with public web research. Implement separate systems for private analytics and public market research. Request privacy documentation from your AI vendors specifically addressing mosaic effect risks.

For Amazon Sellers: Consider that your competitor analysis agents may inadvertently leak your own sales data through query patterns. Use aggregation layers that present anonymous data to public-facing research tools. Monitor query logs for patterns that could reveal proprietary information.

For AI Developers: Integrate privacy metrics into your training objectives from the start. Do not optimize for task performance alone—measure and penalize leakage at the architecture level. Implement PA-DR-style reinforcement learning approaches that jointly optimize accuracy and privacy.

For SaaS Founders: Build privacy-aware features into your product roadmap. The MosaicLeaks research demonstrates that privacy is a competitive differentiator. Companies that can demonstrate measured, verifiable privacy protection will win enterprise trust and compliance approvals.

For Content Marketers: Be aware that AI tools used for content strategy research may leak internal campaign performance data. Establish clear protocols for what information can be shared with AI agents and what must remain in isolated systems.

For Video Creators: When using AI video generation platforms that combine proprietary product data with trend research, verify that the platform employs query sanitization and privacy controls. Request transparency about how your data flows through the system.

FAQ

What is the mosaic effect in AI privacy? The mosaic effect occurs when individually harmless pieces of information, collected from multiple queries or sources, combine to reveal sensitive private data that was not intentionally disclosed. In MosaicLeaks, an agent's benign web searches collectively leak proprietary enterprise information.

How does MosaicLeaks measure privacy leakage? MosaicLeaks measures three levels of leakage: intent leakage (inferring research goals from queries), answer leakage (answering private questions using query log), and full-information leakage (discovering new private facts solely from query logs).

Can simple privacy prompts stop leaks? No. The MosaicLeaks research found that adding a privacy instruction to the system prompt provided inconsistent protection across models, often reduced task performance, and left significant leakage rates of 25% or higher.

Does better AI performance mean more privacy risk? Yes, paradoxically. Training agents exclusively to improve task performance increased answer leakage from 34% to 43% in the MosaicLeaks experiments, because agents made more aggressive web queries to find answers faster.

What is Privacy-Aware Deep Research (PA-DR)? PA-DR is a reinforcement learning approach developed by ServiceNow researchers that jointly optimizes for chain accuracy and privacy preservation. It reduced leakage from 34% to 9.9% while improving task performance from 48.7% to 58.7% strict chain success.

How can ecommerce businesses protect against mosaic effect leaks? Implement architectural isolation between private and public data systems, use query sanitization to remove private identifiers from web searches, deploy automated query log monitoring, and choose AI vendors who demonstrate privacy-aware training methodologies.

References

Sources

Try VEONIB

VEONIB automatically transforms a product URL into comprehensive product analysis, video scripts, storyboards, image prompts, video prompts, and AI-generated marketing videos designed for ecommerce. Visit VEONIB to learn how privacy-aware AI video generation works for your business.

Credibility Assessment

The factual information in this article about MosaicLeaks benchmark design, experimental results, leakage measurements, and training methodologies comes directly from the ServiceNow research published on Hugging Face. The VEONIB analysis regarding implications for ecommerce AI deployments, recommendations for specific merchant types, and AI video workflow applications represent independent interpretation. The comparison table between mitigation approaches synthesizes multiple data points from the original research. The privacy risk scenarios for ecommerce are extrapolated from the general research findings. No information in this article has been fabricated, and any statements about specific model performance reflect the published experimental results.