A/B Testing

jeroendesloovere's Avatar

jeroendesloovere

22 May, 2012 01:08 PM

Something to think about is a way to integrate A/B Testing into Fork.

f.e.: could be implemented as followed for pages...

  • Pages > click on a page you want to test

  • "Settings tab" > checkbox for A/B testing > check it

  • Now select from a dropdown (that appears) the amount of duplicated pages you want (1, 2, 3, 4, 5).

  • Some new tabs will be generated (after save as draft) => 'content', 'content-2', 'content-3', ... These tabs are a copy of the 'content' tab, so you can change the content-template for this duplicate page to the template you want...

You also have to install a new module, "ab_testing" f.e.
In this module it should be able to:

  • set 'click' parameters

  • statistics, ...

  • amount of users to send to version A, B, C, ...

f.e. This could also be implemented for form_builder, to test multiple forms

Let me know what you think about this idea or any other ways to do this.

  1. Support Staff 1 Posted by tijs on 07 Aug, 2012 08:21 PM

    tijs's Avatar

    A/B testing would be nice, not sure if you idea will work for pages.
    If I had to implement it myself, I think I would hack url.php and detect if the page has AB-testing enabled.
    In the module I would set one url and let the admin select two (or more) urls. One for each version to test.

  2. 2 Posted by jeroendesloover... on 07 Sep, 2012 01:23 PM

    jeroendesloovere's Avatar

    Yep, we need to take a deeper look into this...

  3. Support Staff 3 Posted by mlitn on 30 Jan, 2013 10:31 AM

    mlitn's Avatar

    It's quite a hard one to implement, since it could span multiple modules (but we don't want to touch these modules themselves and keep everything nicely separated)

    A perfect A/B-test module, to me, would look like this:

    • It should be possible to create multiple campaigns. Such campaign is then an "extra" that can be attached to a position on a page (e.g. the way it is possible to link a location, or contentblock, ... to a page)
    • Per such campaign, in the A/B-test module, one could create multiple "versions", for which each can get a percentage to be displayed at, ...
    • [Advanced] Such "version" could look like a simplified pages: it could be possible to chose from a certain view per version, and attach multiple widges/editors. Upon page execution, all of these would then be executed upon pageview (depending on the selected version), and the combined output would then be assigned to pages, like any other widget (which means all of the code to handle this should be in the A/B-test module)
    • [Simple] For an initial version though, I wouldn't make it too complex and instead of pages-like functionality, I'd look more in the direction of contentblocks: just 1 editor per version, where it is possible to choose a different .tpl per editor.

    Evaluation of which A/B version is most effective, should not be in Fork CMS, since these evaluation criteria are usually too specific. All we need to provide is a lot of flexibility to be able to create and display multiple versions. Analysis of which version converts better, will likely be done with better tools, like Google Analytics.

    Opinions? Is my rationale clear?

  4. Support Staff 4 Posted by Dieter W on 30 Jan, 2013 01:13 PM

    Dieter W's Avatar

    A suggestion for such a tool is Visual Website Optimizer (http://visualwebsiteoptimizer.com). Other systems have already a great integration with this tool so it should be possible for fork cms too.

  5. 5 Posted by jeroendesloover... on 30 Jan, 2013 02:50 PM

    jeroendesloovere's Avatar

    Another tool could be "Google Content Experiments", integrated in Google Analytics.

    Integration note when we should use this tool:
    in step 3/4 you have to "copy/paste" a GA code on the page (just below the GA starting code).
    So we should then have a textfield in the AB_testing module for this code.

  6. 6 Posted by Dries on 21 Feb, 2013 03:01 PM

    Dries's Avatar

    Visual Website Optimizer only uses a code in the header. The rest is up to the tool.
    For this application, there is only a field necessary to add this code in the header. Best to place this code BEFORE the Google Analytics code.

    Google Content experiments is no issue but this (and other A/B tools) use a more basic approach of copying the page and changing details. You actually have to copy a page manually and change certain blocks in the content (colours, text, placement, ...)

    Note that there is a very big difference in technique here.
    VWO uses Javascript to make adjustments on the page and makes a second variant of the page by adding a query to the page and injecting the changes in the design.
    Google Experiments is fully manually to adapt. No JS involved. Manual coding to make changes.

    I think you need to differentiate the type of A/B test you are letting people to make with fork.
    - Contextual (text) - Design (colours, fonts) - Slice/Dice (?) (placement of content-blocks)

    The most used A/B testing in real-world examples are basic adjustments like:
    - removing content blocks to focus more on forms - removing/adding fields of a form These are often used to make a big impact.

    Test like:
    - Other texts - Changing titles - Changing colours of titles etc These are often more detailed tests

    Note: Call-to-action buttons are in a category between these two mentioned above.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac