A/B testing

Test different thumbnails, whole videos or the wording of a call to action against each other. Pellio splits a video’s viewers evenly between the original and up to three variants, everywhere the video is embedded, and shows which does better. The embed code doesn’t change. A/B testing is on the Advanced plan.

What you can test

• Thumbnail: upload alternative images. Measured by play rate, the share of loads where the viewer pressed play.
• Video: play a different video (another cut, a shorter version, a new intro) in the same embed. Measured by engagement, the share of loads where the viewer watched at least half.
• Call to action: change the heading, text, button or link of one email form, call to action, card or end screen. Measured by conversions, the share of loads with a click or form submission.

You can change the goal of any test: for example, judge a video test by conversions.

Start a test

1. Open the video and go to the A/B test tab.
2. Choose what to test and the goal.
3. Set up each variant: upload its thumbnail, choose its video, or write its new wording. Fields you leave empty in a call to action variant keep the original’s wording.
4. Leave Pick the winner automatically on, or turn it off to decide yourself.
5. Click Start test. It applies straight away to every embed, the share page and WordPress.

A video can have one test running at a time.

Reading the results

For each variant you see its loads (times the player appeared on a page), play rate, the share that watched half, conversions, the change against the original, and its chance to be best: how likely it is, given the data so far, that this variant really is the best on the test’s goal. Small numbers are uncertain, so expect the chance to stay near an even split until each variant has a good number of loads.

Each browser keeps seeing the same variant for 30 days where it allows the player to remember it. Watching the video in Pellio itself doesn’t count toward the results.

Picking the winner

With Pick the winner automatically on, Pellio checks every hour. Once every variant has at least 100 loads and one has a 95% chance of being the best, the test ends and the winner is used. You can also click Use next to any variant, or Keep next to the original, at any time.

• A winning thumbnail becomes the video’s thumbnail.
• A winning call to action’s wording replaces the original’s.
• A winning video keeps playing wherever the original is embedded. Delete the test to go back to the original.

Ended tests stay on the tab with their results until you delete them.

Good to know

• While a thumbnail test runs, the WordPress plugin (0.6.23 and later) loads the player straight away instead of showing a thumbnail first, so each viewer sees their variant’s thumbnail. It can take up to an hour to switch after you start a test.
• Popover embeds show the video’s own thumbnail before they open, so thumbnail tests count them only once the player opens.
• Linked YouTube and Vimeo videos can’t be tested.
• Over the API, GET /v1/videos/{id}/ab-tests returns the results, and AI assistants connected through the MCP server can read them.