Four months ago, xAI promised to stop Grok from generating nonconsensual sexualized images of real women. This week, NBC News reported it is still happening. The bypasses are not sophisticated. Users pair a photo of a real person with a stick-figure pose diagram and tell Grok to “match the pose.” Or they ask Grok to swap the clothing between two images. Or they upload a photo and ask for a video transformation. The filters xAI promised do not catch any of it.
One independent analyst now believes Grok produces more nonconsensual synthetic nudity than every comparable tool combined.
xAI’s publicly described controls amount to model-level filtering — and the company is now arguing in a Dutch court that it cannot stop all abuse and should not be penalized when malicious users bypass those controls. That is the opposite of defense in depth.
Defense in Depth Is Not a Slogan
Defense in depth is a design principle. It assumes every control will fail.
You layer perimeter, network, endpoint, identity, monitoring, and response so that when one layer is breached, the next catches what got through. Each layer is a different control against a different failure mode, often from different tools or different teams. That is the architecture. A single filter is not.
xAI’s Dutch court argument fails a basic test of secure design. CISA’s Secure-by-Design principles place responsibility for safety on the system’s operator, not on end users. Arguing that malicious users are responsible when controls are bypassed does not meet that bar.
Grok and the OWASP LLM Top 10
The OWASP Top 10 for LLM Applications (2025) is the industry reference for critical risks in LLM-based systems. Grok’s public behavior maps directly onto one category — and exposes a gap in the framework itself.
LLM01: Prompt Injection. Prompt injection has held the top spot in the OWASP list for two consecutive editions because LLMs process instructions and data in the same channel without clear separation. The model cannot tell them apart.
This is not a Grok-specific problem. A 2025 paper introduced Cross-modal Adversarial Multimodal Obfuscation (CAMO), a black-box attack framework that splits harmful instructions into benign-looking textual and visual clues. Each component looks harmless on its own. The model reconstructs the attack intent through cross-modal reasoning. CAMO achieved attack success rates of 81.82% on GPT-4.1-nano and 93.94% on DeepSeek-R1 using 12.6% of the tokens required by older attack methods.
The Grok bypasses exploit the same vulnerability class that paper documented — individually benign inputs that become harmful in combination. The difference is that CAMO uses automated adversarial optimization. Grok’s users did not need any of that. They combined unmodified photos with hand-drawn diagrams and plain-language instructions. The filters failed against a basic, manual version of a well-documented attack class. The class itself was publicly documented before xAI shipped these features — Shayegani, Dong, and Abu-Ghazaleh published compositional cross-modality attacks at ICLR in 2024, based on work from 2023.
Where the framework stops. The Grok case also involves insufficient content filtering on generated output and capabilities shipped without proportionate controls. These are real failures, but they do not map cleanly onto a second OWASP LLM Top 10 category. LLM05 (Improper Output Handling) addresses output passed to downstream systems without sanitization — XSS, SQL injection, remote code execution — not harmful content shown directly to users. LLM06 (Excessive Agency) addresses agents calling functions and extensions, not generative models producing content. The OWASP LLM Top 10 was designed for LLM applications integrated into software systems. Consumer-facing generative AI — where the output is the product — sits partially outside the framework's current scope.
The Stakes Get Higher with Agents
Grok generating an image is the low-stakes version of this problem. The failure mode is bad output. When this class of model gets agency — tools, memory, authority to take action — the failure mode stops being bad output and starts being bad actions.
The OWASP Top 10 for Agentic Applications (2026), released in December 2025, is the framework for that next-stage problem. It was built with dozens of security experts from industry, academia, and government and is based on real attacks observed in production.
Agent Goal Hijack (ASI01). An attacker changes an agent’s objectives through malicious content. The same prompt injection that bypassed Grok’s image filters can hijack an agent into sending an email, modifying a record, or calling an API on an attacker’s behalf.
Identity and Privilege Abuse (ASI03). An AI agent acts with the full authority of every key, token, and service account assigned to it. A single agent merges multiple permissions into one execution point. Compromise the agent, and you inherit every non-human identity it holds. Identity runs through most of the top risks in the OWASP Agentic Top 10.
Cascading Failures. A compromised agent does not produce one bad output and stop. It chains actions across connected systems. It exfiltrates through the same channels it was authorized to use.
The model is not your security boundary. The model — and everything you let it do — is the thing being contained.
When the model shares its blast radius with production systems, shares its identity with the user, shares its network egress with sensitive data — you have not deployed AI safely. You have deployed a Grok-class failure waiting for the right prompt.
Five Categories of AI Safety Controls — And Why No Single Category Is Enough
Defense in depth requires controls at different layers, using different methods, catching different failure modes. In AI safety, those controls fall into five categories. Whatever xAI deployed, the publicly visible bypasses on X confirm it was not enough. Here is what the full surface looks like, and where each category fails when the inputs are multimodal.
Category 1: Model-Level Controls
Safety training built into the model itself. RLHF alignment, refusal training, Constitutional AI, concept erasure — techniques that modify the model’s weights to make it refuse harmful requests or suppress harmful outputs.
This is what most people mean when they say “the model won’t do that.”
Model-level controls are useful but have the best-documented failure rates of any category. A 2026 survey of LLM jailbreaking found that automated attacks achieve 90–99% success on open-weight models, and 80–94% on proprietary models. The model cannot reliably separate instructions from content. That limitation is structural. RLHF hasn't fixed it.
Model-level controls are deliberately absent from the six-layer architecture below. The architecture assumes this layer will fail and builds everything else to catch what gets through.
Category 2: Input Inspection
Everything that evaluates the prompt before the model processes it. Prompt injection classifiers, jailbreak detectors, topic deny lists, PII detection on inputs, input format validation.
Available implementations include Azure Prompt Shields, Meta Prompt Guard, NVIDIA NeMo jailbreak detection rails, and Amazon Bedrock’s prompt attack filtering.
Where this category breaks in multimodal: Input inspection for text is a maturing control. The multimodal version is not. The problem is compositional attacks — inputs that are individually benign but harmful in combination. The “Jailbreak in Pieces” paper showed that pairing adversarial images with generic textual prompts breaks model alignment using only the vision encoder — no access to the LLM required.
The Grok bypasses are a simpler version of this attack class. The research attacks use adversarially optimized images. Grok’s users did not need that — they combined unmodified photos with hand-drawn diagrams and simple instructions. The filters failed against an unsophisticated version of a well-documented attack.
Category 3: Output Evaluation
Everything that evaluates the model’s response before it reaches the user. Content harm classifiers, LLM-as-judge implementations, NSFW image classifiers, PII redaction, groundedness checks, output format validation.
Content harm classification is the most widely deployed control in this category — present in every major platform. Azure AI Content Safety monitors four harm categories with adjustable severity thresholds. Amazon Bedrock Guardrails reports blocking up to 88% of harmful content. These classifiers detect harmful outputs when the harm is visible in the output itself. They do not detect harm that was invisible in the inputs and only emerged during generation.
Groundedness checks — verifying that the model’s output is based on provided source material — are shipped by Azure and Bedrock. These address accuracy, not content safety.
Where this category breaks in multimodal: For text, LLM-as-judge works well when the judge is purpose-trained for safety evaluation. For images, the judge needs to be vision-capable and safety-trained on visual content. Few purpose-built visual safety judges exist — Llama Guard 3 Vision and ShieldGemma 2 are among the first. The effectiveness gap is measurable — the best-performing vision classifier in benchmarking studies shows F1 scores below 0.5 on categories like harassment and self-harm.
For video, the problem gets worse. The judge has to evaluate motion, context, and transformation across frames. This is the modality where Grok generates its most harmful output — photo-to-video transformations that are publicly visible on X, meaning whatever output evaluation exists in xAI’s pipeline did not prevent them from reaching users.
Three failure modes in LLM-as-judge are documented.
First, shared blind spots. When the judge and the generator share training lineage, they share failure modes. Research by Fu and Liu (EMNLP 2025 Findings) evaluated five models across 25 languages and found average inter-judge agreement at a Fleiss’ kappa of approximately 0.3 — barely above chance. Liu et al. (ICLR 2025) found that some guard models flag responses as “unsafe” based on the user input alone, even when the model response is a single space token — meaning the guards are classifying the prompt, not the response.
Second, judge vulnerability. The judge is still a model. The same prompt injection techniques that compromise the primary model can compromise the judge. A 2026 survey found that automated judge agreement varies 70–93% depending on implementation.
Third, incomplete coverage. If cost constraints lead to evaluating a sample of outputs rather than all of them, the result is a statistical defense, not a security defense. An attacker who knows that not every output is checked can adjust accordingly.
Category 4: Infrastructure Controls
The controls around the model, not on it. Blast radius containment, network segmentation, identity federation, credential scoping, sandboxed execution, API rate limiting, data loss prevention, egress filtering.
This is the category where existing security expertise applies directly to AI deployment. Zero-trust architecture, least-privilege access, tenant isolation — these are not AI-specific. They are the same controls enterprises have used for decades, applied to a new class of system.
In multimodal: Infrastructure controls are modality-agnostic. They do not care whether the model generates text, images, or video. They care whether the model has access to systems it should not, and whether a compromise propagates to connected systems.
Category 5: Observability
Runtime monitoring, behavioral detection, logging, audit trails, alerting, and incident response.
This category assumes the first four have failed. Runtime monitoring watches for anomalous model behavior — outputs that deviate from baselines, unusual tool invocations, data access patterns outside the agent’s scope. Logging makes incident reconstruction possible. Alerting and incident response make it actionable.
In multimodal: Observability for AI systems is less mature than for traditional infrastructure. Most enterprises have monitoring for network traffic, endpoint behavior, and application logs. Few have equivalent monitoring for AI agent behavior or output distribution anomalies. The telemetry exists — model inputs, outputs, tool calls, guardrail triggers — but it is not routinely fed into SIEM platforms or monitored by security operations centers. The data is available. The pipelines to use it are not built yet.
A Six-Layer Architecture
The six-layer architecture is built from these five control categories, plus one precondition. Model-level controls (Category 1) are not a layer. The architecture assumes they will fail and builds everything else to compensate.
Layer 1: Supply chain visibility (AIBOM). You cannot secure what you cannot inventory. Model provenance, training data origin, fine-tuning history, embedded safety controls, evaluation artifacts. A precondition for evaluating every layer that follows. Maps to LLM03.
Layer 2: Input defense. Category 2 applied. Pre-model classifiers that flag bypass patterns, adversarial inputs, and known-bad prompts. For multimodal systems, classifiers that evaluate the composition of inputs — not each input in isolation. Maps to LLM01 and Agent Goal Hijack.
Layer 3: Output defense. Category 3 applied. Post-model classifiers for every modality the system produces. This layer must use a different detection method than Layer 2. If both share training data or vendor lineage, they share blind spots. Output filtering should be structurally independent: a different model family, a rule-based policy engine, or an LLM-as-judge from a separate provider.
Layer 4: Blast radius and exfiltration controls. Category 4 applied. The model does not share identity with the user. It does not share network egress with production data. It does not share credentials with other agents. Tools are scoped. Permissions are least-privilege. Agent actions are sandboxed. Maps to Identity and Privilege Abuse and Cascading Failures.
Layer 5: Runtime monitoring. Category 5 applied. Layers 1 through 4 try to prevent bad outcomes. Layer 5 assumes they failed. It watches for anomalous behavior, logs everything, and alerts on deviations. This is the layer that catches attacks no classifier was trained on. Logging here is not optional — it is what makes incident reconstruction possible.
Layer 6: Human oversight and incident response. Category 5 extended into action. For high-risk outputs — image generation involving real people, video generation, agent actions that modify production systems — a human review gate belongs in the pipeline. Not on every output. On outputs that cross a defined risk threshold. Behind that gate sits an incident response process: escalation paths, containment procedures, credential revocation, system isolation.
Every layer is imperfect. That is the point. If you can't answer what happens when one layer fails, you don't have defense in depth.
What the Industry Is Shipping Today
Purpose-Built Multimodal Safety Classifiers
Llama Guard 4 (Meta, 2025) is a multimodal safety classifier that evaluates prompts and responses across 14 hazard categories plus code interpreter abuse. Llama Guard 3 Vision (Meta, late 2024) was the first safety classifier built for LLM image understanding, evaluating prompt text and images together. ShieldGemma 2 (Google) classifies images for sexual content, violence, and gore, and uses its own classifier in reverse to generate adversarial test images — red-teaming-as-training. NVIDIA NeMo Guardrails supports multimodal content safety with GPU-accelerated parallel execution, adding roughly half a second of latency for five parallel guardrails.
Every one of these is a single-layer control. None cover compositional cross-modal attacks. They are pieces of a stack, not the stack.
Enterprise Guardrail Platforms
Azure AI Content Safety provides multimodal moderation, prompt injection detection, groundedness checks, and PII filtering — Microsoft’s documentation notes it is probabilistic and should be treated as a risk reduction tool, not a guarantee. Amazon Bedrock Guardrails filters harmful text and image content, blocks prompt injections, and redacts PII — AWS reports it blocks up to 88% of harmful content. Microsoft Foundry Guardrails applies classification at four intervention points: user input, tool call, tool response, and output — the tool call and tool response points are significant for agentic systems because they let guardrails inspect what an agent is about to do before it does it.
Every one of these platforms is built on classification models tuned for known harm categories. None cover compositional cross-modal attacks. They are a layer. The defense-in-depth architecture has to be built by the enterprise deploying them.
Emerging Research and Open Problems
In-Generation Detection
Current safety tools inspect the prompt or the output. A 2025 preprint introduced In-Generation Detection (IGD), which monitors the model’s internal state during the image generation process itself. It reads the predicted noise during diffusion denoising steps — a signal that reflects the evolving visual meaning of the prompt — and trains a lightweight classifier to detect NSFW intent before the image is fully generated.
IGD achieved 91.32% detection accuracy across seven NSFW categories, including adversarially crafted prompts. Because it reads internal model state rather than the prompt surface, it has the potential to catch adversarial inputs that are designed to look benign to external classifiers.
Currently demonstrated only for diffusion-based image generation. Does not extend to video, text, or multimodal-to-multimodal systems. Not shipping in any enterprise product.
Proposed Directions for Compositional Attack Defense
Security researchers have described three architectural directions that would address the compositional cross-modal attack class:
Evaluate combined intent, not individual inputs. Safety systems should reason over the cumulative meaning of a full prompt sequence — “photo + stick figure + match the pose” as a single semantic intent, not three benign inputs evaluated separately.
Share context across safety layers. The image classifier should see the original user request. The prompt guard should see the generated image. Without this, attackers can route harmful content through one modality to exploit blind spots in another.
Decompose compositional inputs. Classifiers should identify compositional elements — diagrams, reference images, pose guides — within a larger input, and evaluate their meaning separately from the overall scene.
None of these are shipping in enterprise products. They represent where the field needs to go, and security architects should be asking vendors whether their roadmaps address them.
The Swiss Cheese Model for AI Safety
Researchers have proposed multi-layered runtime guardrails modeled on the Swiss Cheese Model from aviation and healthcare safety engineering. Each layer has holes. The principle is that no two layers have the same holes in the same place. The architecture decouples safety authority from any single model so each layer can be tested and updated independently.
What a Security Architect Should Implement Today
The research is ahead of the products. The products are ahead of most deployments. Here is what you can do now, mapped to the six layers, using tools that exist today.
Where you start depends on what you are shipping. If the business needs an internet-facing chatbot, input defense comes first — you need prompt injection detection before the system goes live. If the system handles legal or regulated content, output filtering on specific terms comes first — you need to block what cannot be said before anything else. The layer numbers are not a priority order. They are a completeness checklist. Build what the use case demands, ship it, then add depth.
I'm building this stack in production — some layers are live, others are in progress. Start anywhere, but don't stop at one layer — the gap you skip is the one that gets exploited.
The tooling is also not static. Security vendors are building AI capabilities into their products at the same pace enterprises are adopting AI. The guardrail platform you evaluated last quarter may have shipped new capabilities since. Reassess continuously. And check what you already have — if your organization runs DLP, content filtering, or compliance tooling, some of these controls may already be partially in place. You do not always need to build from scratch.
Layer 1: Supply chain visibility. Maintain an AIBOM for every model in your environment. Document provenance, training data sources, fine-tuning history, safety controls, and evaluation results. For third-party models, document what the vendor discloses and what they do not.
Layer 2: Input defense. Deploy prompt injection and jailbreak detection on all inputs before they reach the model. For multimodal systems, use classifiers that evaluate the composition of inputs, not just individual components. Meta Prompt Guard, Azure Prompt Shields, and NVIDIA NeMo jailbreak detection are available options. None fully solve compositional attacks, but their absence is what lets those attacks scale. Run them in parallel with other guardrails to minimize latency.
Layer 3: Output defense. Deploy a structurally independent output classifier. If your input classifier is from Vendor A, your output classifier should not be from Vendor A. Use a purpose-built multimodal safety classifier — Llama Guard 4, ShieldGemma 2, or comparable — rather than a general-purpose vision model. If your system generates images or video, the classifier must be trained on AI-generated content, not benchmarked against real-world photos. Test it against adversarial inputs, not just known harmful content.
Layer 4: Blast radius and exfiltration controls. Apply your existing zero-trust and least-privilege architecture to AI systems. The model runs in a sandbox. It does not share identity with the user, network egress with production data, or credentials with other agents. Tools are scoped and explicitly enumerated. Rate limits, DLP rules, and egress filtering apply to AI-initiated requests the same way they apply to human-initiated requests.
Layer 5: Runtime monitoring. Log all inputs, outputs, tool invocations, and guardrail triggers. Establish behavioral baselines and alert on deviations. Feed guardrail trigger data into your SIEM. If your SOC monitors network anomalies and endpoint behavior, it should also monitor AI agent behavior.
Layer 6: Human oversight and incident response. Define risk thresholds for human review. Build incident response playbooks for AI-specific scenarios: model compromise, agent hijack, data exfiltration through authorized channels, classifier bypass. Include the ability to revoke agent credentials, isolate the model, and preserve audit logs.
Architecture-level: Run guardrails in parallel, not in series. Five parallel guardrails add roughly half a second of latency. Use risk-based routing — low-risk queries get lightweight checks, high-risk queries get deeper evaluation with human review gates.
The Takeaway
If the safety story for any AI system you build or deploy is “the model won’t do that,” that is a red flag. Ask what catches the prompt the model missed. Ask what catches the prompt that does not look like a prompt. Ask what the classifier’s detection rate is on AI-generated content specifically. Ask what the judge does when the judge is the target.
If your AI adoption strategy treats the model as the security boundary, you are one creative composition away from the Grok headline. Not the same incident. The same failure class.
The security team has decades of defense-in-depth experience. The AI safety field is still building theirs. We have done this before. We know what happens when a single control fails without a second layer behind it.
The answer is not “we could not prevent all abuse.”
The answer is the next layer.
Subscribe to StepToCyber for frequent analysis on securing GenAI at enterprise scale.
Views are my own.
References
Primary news coverage
Ingram, D. (2026, April 14). Elon Musk’s AI chatbot Grok continues to produce sexualized deepfakes despite xAI’s pledge to stop. NBC News. https://www.nbcnews.com/tech/tech-news/musks-ai-chatbot-grok-xai-making-sexual-deepfakes-imagine-rcna265855
OWASP frameworks
OWASP GenAI Security Project. (2025). OWASP Top 10 for Large Language Model Applications 2025. https://genai.owasp.org/llm-top-10/
OWASP GenAI Security Project. (2025, December). OWASP Top 10 for Agentic Applications 2026. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
Secure-by-Design
CISA. Secure by Design. https://www.cisa.gov/securebydesign
Compositional cross-modal attack research
Shayegani, E., Dong, Y., & Abu-Ghazaleh, N. (2024). Jailbreak in Pieces: Compositional Adversarial Attacks on Multi-Modal Language Models. ICLR 2024. https://openreview.net/forum?id=plmBsXHxgR
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models (CAMO). (2025). arXiv preprint. https://arxiv.org/html/2506.16760v1
Image safety classifier benchmarking
Qu, Y., Shen, X., He, X., Backes, M., Zannettou, S., & Zhang, Y. (2024). UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images. arXiv preprint. https://arxiv.org/html/2405.03486v3
LLM jailbreaking survey and judge reliability
Bin Hakim, S., Gharami, K., Farhady Ghalaty, N., et al. (2026, January). Jailbreaking LLMs: A Survey of Attacks, Defenses and Evaluation. TechRxiv. https://www.techrxiv.org/users/1011181/articles/1373070
Fu, X. & Liu, W. (2025). How Reliable is Multilingual LLM-as-a-Judge? EMNLP 2025 Findings, pages 11040–11053. https://aclanthology.org/2025.findings-emnlp.587/
Liu, H., Huang, H., Gu, X., Wang, H., & Wang, Y. (2025). On Calibration of LLM-based Guard Models for Reliable Content Moderation. ICLR 2025. https://arxiv.org/abs/2410.10414
Purpose-built multimodal safety classifiers
Meta. (2025). Llama Guard 4-12B Model Card. Hugging Face. https://huggingface.co/meta-llama/Llama-Guard-4-12B
Meta. (2024). Llama Guard 3-11B-Vision Model Card. GitHub. https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/11B-vision/MODEL_CARD.md
Google. ShieldGemma. Referenced in: 19 Large Language Models Redefining AI Safety. InfoWorld. https://www.infoworld.com/article/4140809/19-large-language-models-redefining-ai-safety-and-danger.html
Enterprise guardrail vendor documentation
Microsoft. (2026). Azure AI Content Safety overview. https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety/
Microsoft. (2026). Guardrails and controls overview in Microsoft Foundry. https://learn.microsoft.com/en-us/azure/foundry/guardrails/guardrails-overview
Amazon Web Services. (2026). Amazon Bedrock Guardrails. https://aws.amazon.com/bedrock/guardrails/
NVIDIA. NeMo Guardrails for Developers. https://developer.nvidia.com/nemo-guardrails
Guardrail architecture research
Designing Multi-layered Runtime Guardrails for Foundation Model Based Agents: Swiss Cheese Model for AI Safety by Design. (2024). arXiv preprint. https://arxiv.org/html/2408.02205v3
Modular Safety Guardrails Are Necessary for Foundation-Model-Enabled Robots in the Real World. (2026). arXiv preprint. https://arxiv.org/html/2602.04056
In-generation detection research
Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models. (2025). arXiv preprint 2508.03006. https://openreview.net/forum?id=SFHjSDIMKn
Proposed architectural defenses
Decodes Future. (2026, March). Grok Jailbreak Prompts: Multimodal Reasoning Vulnerability Analysis. https://www.decodesfuture.com/articles/grok-jailbreak-prompts-multimodal-reasoning-vulnerability-analysis
Parallel guardrail orchestration
Authority Partners. (2026, March). AI Agent Guardrails: Production Guide for 2026. https://authoritypartners.com/insights/ai-agent-guardrails-production-guide-for-2026/