Attentive | A/B Testing

For our brands that use Attentive for their pop up and SMS, almost 100%, if not 100% use Klaviyo for emails. This guide will help you not only in setting up the A/B test within Attentive, but will walk through what's needed to configure the test in the brand's Klaviyo email flow.

Part One: Update your Klaviyo flow


In order for the Attentive pop up to configure with the Klaviyo email flow, you must ensure the conditional split at the top of the email flow is set up properly. Attentive can assign Klaviyo source tags to the pop ups, which will correspond with the conditional split you set up. Here's how to do this:

1. Click Flows > locate appropriate flow

  1. Go to Logic > Conditional Split > click and drag underneath flow trigger
  2. Under Select a Condition > Properties about someone
  3. Under Dimension > $source

IMPORTANT: $source is the dimension Attentive uses to set a Klaviyo tag to your Attentive pop ups. If a different dimension is selected (ex: source, Source, initial source, etc.) this entire set up will not work.

  1. Click does not equal
    1. As a fail safe, we use "does not equal" in the event that an A/B test is accidentally set up incorrectly or CashBack doesn't work. The dimension we enter will always be the CashBack Klaviyo source tag. That way, a shopper will still receive an offer from the brand if CashBack isn't configured properly for whatever reason.
  2. Under Dimension Value > enter Klaviyo source tag you will give to Attentive

IMPORTANT: The dimension value you enter here is what you will provide to Attentive to connect the pop up to this flow.

    1. For consistency sake, please use the following format as your dimension value: attentive(offer) ex: attentive10off, attentive20back
  1. Click Save

Part Two: Set Up A/B Test on Attentive

  1. Chat with Attentive White Glove Service

Attentive requires A/B tests to be set up directly by their White Glove Service. Please click the Live Chat button in the top right corner of your Attentive screen and use the following messages to set up A/B tests and connect them to your Klaviyo flow.

IMPORTANT: While you can email Attentive with these requests, it is not recommended. There have been several instances where Attentive hasn't configured the test properly, so it's best to catch this as the test is going live rather than at a later time. We want to have total control of this A/B test, including when it goes live rather than when an Attentive rep gets to your email.

Message 1: Hi! Can I please have help setting up A/B tests and Klaviyo source tags for my sign up units? 

Their team will likely ask you to confirm account information as well as ask about what units you'd like to A/B test.

IMPORTANT: You do not want them to change the Klaviyo list this sign up unit is submitting to--you only want to add a Klaviyo source tag to the sign up unit.

You must provide separate Klaviyo source tags for the sign up units based on if they are for a discount or CashBack. Even though only one source tag is mentioned within the conditional split set up in the Klaviyo email flow, it's important to have source tags set for all sign up units. This will help with data reporting.

It's easiest to provide their team with the Creative ID assigned to the unit, which can be found here:

Message 2a (Adding Source Tags): 

Can the following units have a Klaviyo source tag of "attentive10off" set:

  1. 331123
  2. 331124

Can the following units have a Klaviyo source tag of "attentive20back" set:

  1. 331125
  2. 331126

Message 2b (Setting up A/B Tests): 

Once complete, can the following units be A/B tested against each other: 

  1. 331123 vs. 331125
  2. 331124 vs. 331126

After they complete this set up, they'll confirm all tests and Klaviyo source tags are set. Once the tests are live, you can set the email series live. 

IMPORTANT: Once the test is live, test ALL arms of the A/B test to ensure you are receiving the proper messaging based on the pop up you signed up for. Attentive is very closed handed in what they allow you to set up, so since we don't have control over setting up the A/B test, it's vital to check their work.