Variolab – A/B Testing

Run A/B tests by pointing the plugin at two existing pages — one as the control (Variant A), one as the variant (Variant B). Visitors are split 50/50 via a persistent cookie; once assigned, they always see the same variant.

Tracking is fully internal: impressions and conversions land in a custom database table, and the wp-admin dashboard shows conversion rates, lift, and a basic statistical significance indicator (two-proportion z-test).

Security-audited internally before every release (situated checklist + OWASP grid). See SECURITY.md on GitHub for the disclosure policy and the latest audit report (docs/security/latest.md).

Features

• Page-level A/B tests (entire page as variant — no Gutenberg surgery needed)
• Persistent cookie split (httponly, samesite=Lax)
• Internal tracking — no third-party dependency, no data leaving your site
• Conversion goals: URL visited or CSS selector clicked
• Auto bypass for logged-in editors and bots
• Two-proportion z-test for statistical significance
• abtest_event_logged action hook ready for v2 GA4/webhook integrations

Caching

A/B testing breaks under page caching: the first variant served gets cached for everyone, all subsequent visitors get that same response, the 50/50 split dies. The plugin handles most cases automatically.

What the plugin does automatically

1. Sends Cache-Control: no-store headers on every page response under A/B test. Respected by Cloudflare, Varnish, Kinsta edge cache, nginx page cache, and most server-level caches.
2. Hooks WP Rocket’s rocket_cache_reject_uri filter when WP Rocket is detected — your test URLs are auto-added to the never-cache list.
3. Hooks LiteSpeed Cache’s litespeed_force_nocache_url filter when LiteSpeed is detected — same idea.
4. Surfaces an admin notice when a cache plugin or known host (like Kinsta) is detected, with what to verify.

Hosting on Kinsta

Kinsta uses a two-layer cache (nginx server-cache + Cloudflare Enterprise edge cache). The plugin’s no-store headers bypass both — but for 100% safety, also add your test URLs to MyKinsta → Tools → Cache → Cache Bypass as URL Patterns (e.g. ^/promo/$). After publishing a new test, purge the Kinsta cache to flush any version cached before the experiment started.

Verify it works by inspecting headers on your test URL:

Look for X-Kinsta-Cache: BYPASS (or MISS). If you see HIT, you’re getting the cached version and the split is broken — purge the cache.

Hosting on other CDNs / hosts

• Cloudflare APO: Cache-Control headers from origin override the cache. Should work out of the box. Verify with curl -I looking for cf-cache-status: BYPASS or DYNAMIC.
• WP Engine: Add the test URLs to the “Cache Exclusions” list in your User Portal.
• Pagely / Pantheon / Pressable: Cache-Control headers respected. Add manual URL exclusion in the host’s panel for safety.

Plugins not auto-supported

For W3 Total Cache, WP Super Cache, WP Fastest Cache, and Cache Enabler — the plugin shows a notice but does not automatically exclude URLs (no clean public API). Manually add your test URLs to that plugin’s cache exclusion list.

REST API

Pull stats programmatically from external tools (n8n, Make, Pipedream, dashboards).

• Endpoint: GET /wp-json/abtest/v1/stats
• Auth: WP Application Passwords (Basic Auth). The user must have manage_options.
• Generate one in your WP profile → Application Passwords.

Optional query params:

• url=/promo/ — filter to a single test URL.
• experiment_id=38 — fetch a single experiment by ID.
• status=running|paused|ended|draft — filter by status.
• from=YYYY-MM-DD&to=YYYY-MM-DD — restrict event date range for the stats computation.
• breakdown=daily — include per-day time series (for charting).

Example:

The response includes for each experiment: id, title, test_url, status, dates, control/variant IDs, goal, and a stats block with A/B impressions/conversions/rate, lift, p-value, significance, and 95% confidence interval bounds for both lift and absolute difference.

Privacy

The plugin stores no raw IP, no User-Agent, no email, no name, and no cross-site tracking identifier. The events table contains: experiment_id, variant, test_url, event_type, created_at, and a visitor_hash = first 16 hex chars (64 bits) of sha256(IP + UA + wp_salt('auth')) — non-reversible, single-site, salt-rotated, dedup-safe. Cookies are httponly, samesite=Lax, secure on HTTPS, value = a single letter (a/b/c/d), 30-day TTL.

A native WordPress privacy guide snippet is registered automatically — find it under Settings → Privacy → Policy Guide → Variolab – A/B Testing to paste into your privacy policy.

For consent-banner sites: enable “Require consent” in the plugin settings and wire your banner to the abtest_visitor_has_consent filter (return true to track, false/null to block). Snippets for Complianz, CookieYes, and Cookiebot are in the README on GitHub.

Right to erasure: because no reversible identifier is stored, individual deletion isn’t possible. Use TRUNCATE wp_abtest_events to erase all A/B testing data.

External services

This plugin connects to one external service, only when the site administrator opts in through the plugin’s Settings → Google Analytics 4 panel by entering a Measurement ID and API Secret. With the GA4 integration disabled (default), no data leaves your site.

Google Analytics 4 (Measurement Protocol)

What it is and what it’s used for: when the GA4 integration is enabled, the plugin forwards A/B-test impression and conversion events to Google Analytics 4 via the Measurement Protocol, so the test results can be analyzed alongside your existing GA4 reports.

What data is sent and when: on each impression and each conversion logged by the plugin, a single fire-and-forget HTTPS request is sent to https://www.google-analytics.com/mp/collect with a JSON payload containing:

• client_id — the plugin’s internal visitor hash (truncated salted SHA-256 of IP + User-Agent; never the raw IP or UA)
• events[].name — abtest_impression or abtest_conversion
• events[].params.experiment_id — the WordPress post ID of the experiment
• events[].params.variant — the variant served (a, b, c, or d)
• events[].params.test_url — the URL path under test (e.g. /promo/)

No raw IP address, User-Agent string, email, name, WordPress user ID, or page content is sent.

Service provided by Google. Please review their terms and policies before enabling the integration:

• Google Analytics terms of service: https://marketingplatform.google.com/about/analytics/terms/us/
• Google privacy policy: https://policies.google.com/privacy