Skip to main content

Technical Architecture

How VendorTruth actually works under the hood—what's real, what's aspirational.

The Problem We're Solving

B2B vendors make bold assertions buyers can't easily verify independently:

  • "Zero vendor lock-in"
  • "Elastic scalability"
  • "Enterprise-grade security"
  • "Transparent pricing"

The challenge:

  • Vendor marketing emphasizes benefits, downplays limitations
  • Buyers lack time/resources to adversarially research every claim
  • Discovery of gotchas often happens post-purchase

What VendorTruth does: Accelerates adversarial vendor research by automating what skeptical buyers would manually investigate (Google for problems + Google for benefits).

What it doesn't do: Replace customer references, POCs, or legal review. Supplements manual research, doesn't eliminate it.

How Dialectical Verification Works

The methodology:

  • System spawns two AI agents with opposing research goals
  • Agents run in parallel (not sequential) for 30-60 seconds each
  • Each agent recursively explores its perspective 1-5 levels deep
  • Final synthesis combines both perspectives into balanced verdict

Limitations:

  • Both agents use the same language model and search API (Exa)
  • Not truly adversarial like courtroom (no rebuttal phase, no cross-examination)
  • Synthesis is AI-generated, not human-judged

Bottom line: Parallelizes pro/con research to surface balanced evidence faster. Automates what skeptical buyers manually do (Google for problems + Google for benefits).

The Actual Process

When you verify a vendor claim, here's what happens:

1. Prosecution Agent (The Skeptic)

What it does:

  • Searches for evidence challenging the claim (hidden costs, limitations, failures)
  • Generates follow-up questions focusing on gotchas and risks
  • Prioritizes sources: customer complaints, migration stories, critical reviews

What it doesn't do:

  • Access private communities (internal Slack, vendor support tickets)
  • Conduct original research (benchmarks, security audits)
  • Interview actual customers (relies on public testimonials only)

2. Defense Agent (The Advocate)

What it does:

  • Searches for evidence supporting the claim (innovations, advantages, success stories)
  • Generates follow-up questions focusing on strengths and use cases
  • Prioritizes sources: vendor docs, case studies, positive reviews

What it doesn't do:

  • Verify vendor claims independently (relies on vendor-provided evidence)
  • Distinguish genuine innovations from marketing hype algorithmically
  • Access insider information (private beta features, roadmap details)

3. Synthesis (The Judge)

What it does:

  • Combines findings from both agents into structured report
  • Assigns verdict rating (True / Mostly True / Misleading / False / Unverified / Mixed)
  • Extracts strengths, weaknesses, recommendations from evidence

What it doesn't do:

  • Human fact-checking (pure AI synthesis)
  • Resolve conflicting evidence algorithmically (reports contradictions, doesn't arbitrate)
  • Provide legal/compliance validation of findings

Example Research: "AWS has zero vendor lock-in"

What actually happened during verification:

Prosecution Agent found:

  • Proprietary services (RDS, Lambda, DynamoDB) lack direct equivalents on other clouds
  • Data egress fees ($0.09/GB) create financial barrier to switching (AWS pricing)
  • Infrastructure-as-Code tools (CloudFormation, CDK) are AWS-specific
  • Organization invested in AWS certifications/training (sunk cost)

Defense Agent found:

  • Terraform provides cloud-agnostic IaC alternative
  • Kubernetes runs portably across clouds (EKS → GKE → AKS migration path)
  • EC2 and S3 have broadly compatible APIs (S3-compatible storage exists everywhere)
  • Strong ecosystem (1000+ integrations) reduces dependency on AWS-only features

Synthesis verdict: Misleading

What's True:

  • Basic compute (EC2) and storage (S3) are relatively portable
  • Open-source tooling (Terraform, K8s) enables multi-cloud architecture
  • AWS doesn't contractually prevent migration

What's False:

  • "Zero" lock-in is absolute claim—false for managed services (RDS, Lambda, DynamoDB)
  • Data egress fees create economic lock-in (not technical, but real)
  • Organizational lock-in (training, expertise, tooling) is underestimated

Recommendation:

  • If using only EC2 + S3 + Terraform → low lock-in risk
  • If using managed services heavily → significant lock-in risk
  • Design for portability upfront if multi-cloud is strategic requirement

What You Get

Comprehensive Truth Reports

Each verification report includes:

  • Executive Summary: Quick verdict and key takeaways
  • Dialectical Analysis: Full prosecution and defense findings
  • Evidence: Direct links to vendor documentation, blog posts, and third-party sources
  • Impact Assessment: What this means for your use case
  • Recommendations: Actionable guidance for decision-making

Continuous Vendor Monitoring

Set up alerts to track changes to vendors you're evaluating or already using:

  • Pricing Changes: New fees, price increases, billing model changes
  • Product Updates: Feature additions, deprecations, or breaking changes
  • Policy Modifications: Terms of service or privacy policy updates
  • Security Advisories: Vulnerabilities, incidents, or compliance issues

Example alert:

"MongoDB Atlas pricing increased by 15% for compute-optimized clusters. Impact: High for data-intensive workloads. Affects M30+ cluster tiers starting March 2025."

Interactive Knowledge Graph

Every truth report becomes part of an interconnected knowledge base:

  • Explore related topics: Click inline links to dive deeper into concepts
  • Compare vendors: See how competing solutions stack up
  • Track trends: Identify patterns across vendor behavior
  • Build context: Understand the broader landscape before deciding

Data Sources & Transparency

Where we search:

  • Public web via Exa API
  • Vendor docs, blog posts, GitHub issues, Stack Overflow, Reddit, reviews
  • Every factual claim links to source URL
  • No made-up sources (hallucinated URLs filtered out)

What we can't access:

  • Paywalled content (Gartner, Forrester analyst reports)
  • Private communities (Slack, Discord, vendor support tickets)
  • Confidential customer feedback
  • Unlisted or login-gated content

Strengths:

  • Transparent sourcing (you can verify claims yourself)
  • Diverse source types reduce single-source bias
  • Public data often sufficient for established vendors

Limitations:

  • New vendors (<6 months) have sparse public footprint
  • We cite published benchmarks, don't run our own tests
  • Vendor-controlled sources may lack critical perspectives

When data sources are adequate:

  • Established B2B vendors with active communities
  • Claims verifiable via public docs (feature support, pricing tiers)
  • Questions with public evidence trail (outages, migrations, reviews)

When data sources are inadequate:

  • Stealth-mode startups with minimal public presence
  • Claims requiring insider knowledge (roadmap timelines, internal architecture)
  • Niche vendors with small communities

Data Freshness & Accuracy

Freshness:

  • Reports generated on-demand (not pre-cached from old data)
  • Research happens during 2-5 minute generation window (fresh as of report timestamp)
  • Monitoring checks run hourly (Pro) or daily (Free)
  • Search results reflect Exa's index freshness (typically 1-7 days lag for new content)

Uncertainty Handling:

  • System explicitly states "insufficient data" when evidence is weak
  • Uncertainty handling is algorithmic (AI judges sufficiency)
  • No human fact-checking layer

Accuracy:

  • ❌ No SLA on factual accuracy
  • ❌ No guarantee reports catch all gotchas
  • ✅ All claims cite source URLs (you can verify)
  • ✅ Explicit "insufficient data" when evidence is weak

When to trust report accuracy:

  • Vendor has extensive public documentation
  • Multiple independent sources corroborate finding
  • Claims link to primary sources (not secondary summaries)

When to be skeptical:

  • Only vendor-controlled sources cited (no independent verification)
  • Evidence is sparse or dated (>12 months old)
  • Contradictory evidence flagged but not resolved

Security & Privacy

Security Measures:

  • API requests scoped to your account (not shared with other users)
  • You control whether reports publish to public knowledge graph (default: private)
  • Verification requests not shared with vendors being researched
  • Data encrypted in transit (HTTPS/TLS 1.3) and at rest (AES-256)
  • Enterprise SSO supported (SAML 2.0, OAuth 2.0)
  • Account isolation prevents cross-user data leakage

What's Not Available:

  • No SOC 2 certification (enterprise procurement may require this)
  • No GDPR third-party audit (we follow GDPR practices but not formally audited)
  • No published penetration testing results
  • No bug bounty program
  • Data retention policy not documented

Important: Subprocessors (Exa API, OpenAI) have access to verification queries. Check their privacy policies separately.

When security/privacy is adequate:

  • Standard B2B SaaS risk tolerance
  • Non-confidential vendor research (public information only)
  • You're OK with subprocessor data sharing (Exa sees your search queries)

When to be cautious:

  • Enterprise procurement requiring SOC 2 Type II (we don't have it yet)
  • Highly regulated industries (healthcare, finance) requiring audited compliance
  • Confidential vendor evaluations where query itself reveals competitive strategy

Integration Options

Web Application

  • Interactive chat interface for verification requests
  • Browse existing truth reports and vendor profiles
  • Manage monitoring alerts and subscriptions
  • Export reports to PDF or Markdown

REST API

  • Programmatic access for automated verification workflows
  • Webhook notifications for monitoring alerts
  • Batch processing for multiple vendor evaluations
  • See Integration Guide for code examples

Browser Extension (Coming Soon)

  • Right-click any vendor claim to verify instantly
  • Inline warnings on vendor websites for known gotchas
  • Quick verdict tooltips without leaving your current page

Use Cases

Pre-Purchase Evaluation

Scenario: Your team is evaluating database vendors for a new project.

VendorTruth workflow:

  1. Verify "elastic scalability" claims for MongoDB, Postgres, and CockroachDB
  2. Compare pricing structures to uncover hidden costs
  3. Check lock-in risk for each option
  4. Get balanced recommendations based on your requirements

Outcome: Make an informed decision backed by adversarial research, not just vendor marketing.

Contract Negotiation

Scenario: Vendor claims "industry-leading uptime" but SLA details are vague.

VendorTruth workflow:

  1. Verify historical uptime claims against public incident reports
  2. Compare SLA terms to industry standards
  3. Identify concerning liability limitations in fine print

Outcome: Negotiate from a position of knowledge with specific data points.

Migration Planning

Scenario: Considering migrating from AWS to Google Cloud to reduce costs.

VendorTruth workflow:

  1. Verify GCP's "20% cheaper than AWS" pricing claims
  2. Identify AWS-specific services that don't have direct GCP equivalents
  3. Estimate true migration costs including engineering time and data egress

Outcome: Realistic migration plan with accurate cost projections, not just sticker price comparisons.

Platform Capabilities

Verification Engine

  • Multi-source research synthesis
  • Real-time evidence gathering
  • Bias detection in vendor marketing
  • Contradiction identification across claims

Monitoring System

  • Automated vendor page tracking
  • Change detection and significance analysis
  • Customizable alert thresholds
  • Multi-channel notifications

Knowledge Graph

  • Interconnected vendor intelligence
  • Topic exploration and discovery
  • Trend analysis across vendors
  • Historical claim tracking

API Access

  • RESTful API for automation
  • Webhook integrations
  • Batch processing
  • Rate limits by plan tier

Performance & Reliability

Report Generation:

  • Comprehensive verification: 2-5 minutes (actual measured median: 3.2 minutes)
  • Single-depth research: 30-60 seconds
  • API endpoint latency: ~150ms p50, ~800ms p99

Limitations:

  • Times assume Exa API availability (downstream dependency failures add latency)
  • No SLA on report generation time (2-5 minute range is typical, not guaranteed)

Infrastructure:

  • Hosted on Vercel (inherits their availability)
  • Multiple edge regions for static content delivery
  • Vercel edge functions auto-scale for concurrent requests
  • Rate limiting by tier prevents resource exhaustion

What's Missing:

  • No uptime SLA (best-effort availability, no formal guarantee)
  • Database is single Postgres instance (no published multi-region failover)
  • Report generation can queue if Exa API is rate-limited
  • No published load testing results

When performance is adequate:

  • Standard vendor research timelines (minutes acceptable)
  • Non-time-critical workflows (async report generation)
  • Low concurrency (< 10 simultaneous users per account)

When to be concerned:

  • Time-sensitive decisions (<5 minute tolerance)
  • High-concurrency scenarios (100+ team members generating reports simultaneously)
  • Enterprise SLA requirements (no formal SLA offered yet)

Pricing

Free Tier:

  • 10 verifications/month
  • Daily monitoring checks
  • Public knowledge graph access
  • Community support (best-effort)

Pro Tier ($49/month):

  • 500 verifications/month
  • Hourly monitoring checks
  • API access with webhooks
  • Priority support (target: 24hr response)
  • Export to PDF/Markdown

Enterprise (Custom pricing):

  • Unlimited verifications (subject to fair use)
  • Dedicated monitoring infrastructure
  • SSO and team management
  • Custom integrations
  • Dedicated support (Slack channel)

What's Not Documented:

  • Overage pricing on Pro tier (what happens at 501 reports?)
  • Fair use definition for Enterprise "unlimited" tier
  • Webhook notification throttling limits
  • Price increase policy
  • SLA terms (no formal SLA offered yet)

When pricing is transparent enough:

  • Free/Pro tier with usage well within limits
  • You're okay with best-effort support
  • No formal SLA required

When to negotiate:

  • Enterprise tier (all pricing is custom anyway)
  • Need formal SLA documentation
  • High volume usage (> 1000 reports/month)

Next Steps