Welcome to the vibrant world of EchoLogyx, where A/B testing meets innovation! Our tailored onboarding process is meticulously designed to align with your needs, ensuring a smooth and efficient start to our partnership. Let’s explore how we make this happen.
Step 1: The Introductory Call
Our journey kicks off with an engaging introductory call . Here, we delve into your project’s unique A/B testing requirements. You’ll be paired with a dedicated team member and manager, offering you personalized support and in-depth expertise from day one.
Step 2: Team Meeting and Process Synchronization
We follow up with a comprehensive team meeting, uniting EchoLogyx’s experts with your stakeholders. This session is key to syncing our structured A/B testing process with your operational practices, ensuring a seamless and effective strategy.
Step 3: Detailed Checklist and Website Analysis
Focusing on the finer points, we discuss your website’s A/B testing needs, employing a thorough checklist to cover every project facet. This ensures we fully grasp and prepare for your specific requirements.
Step 4: Customizing Tools and Templates
Our onboarding includes a deep analysis of your tools and templates, especially for product detail page experiments. We scrutinize your product layouts, promotional techniques, and various templates to tailor our A/B testing strategies precisely for you.
Step 5: Project Management Tool Familiarity
At EchoLogyx, we pride ourselves on being well-versed in a variety of project management tools. Whether you prefer Trello, Asana, Jira, Monday, ClickUp, Teamwork, or Airtable, we’re equipped to integrate seamlessly with your system. This flexibility ensures that all necessary A/B testing information is organized and accessible, fostering an efficient workflow.
Step 6: Consistent Updates and Open Communication
Regular catch-ups and a dedicated communication channel, like Teams, are central to our approach. These ensure continuous, effective communication, vital for monitoring project progress and adapting to evolving needs.
Your A/B Testing, Our Expertise
Client onboarding at EchoLogyx is more than a process; it’s the first chapter in a story of A/B testing triumph. With our commitment to personalized service and adaptive strategies, we’re ready to bring your A/B testing projects to life.
Eager to embark on this journey with us? Click the link in the comments to contact us and schedule a call! We’re excited to collaborate and achieve great results together.
Welcome to the vibrant world of EchoLogyx, where A/B testing meets innovation! Our tailored onboarding process is meticulously designed to align with your needs, ensuring a smooth and efficient start to our partnership. Let’s explore how we make this happen.
Step 1: The Introductory Call
Our journey kicks off with an engaging introductory call . Here, we delve into your project’s unique A/B testing requirements. You’ll be paired with a dedicated team member and manager, offering you personalized support and in-depth expertise from day one.
Step 2: Team Meeting and Process Synchronization
We follow up with a comprehensive team meeting, uniting EchoLogyx’s experts with your stakeholders. This session is key to syncing our structured A/B testing process with your operational practices, ensuring a seamless and effective strategy.
Step 3: Detailed Checklist and Website Analysis
Focusing on the finer points, we discuss your website’s A/B testing needs, employing a thorough checklist to cover every project facet. This ensures we fully grasp and prepare for your specific requirements.
Step 4: Customizing Tools and Templates
Our onboarding includes a deep analysis of your tools and templates, especially for product detail page experiments. We scrutinize your product layouts, promotional techniques, and various templates to tailor our A/B testing strategies precisely for you.
Step 5: Project Management Tool Familiarity
At EchoLogyx, we pride ourselves on being well-versed in a variety of project management tools. Whether you prefer Trello, Asana, Jira, Monday, ClickUp, Teamwork, or Airtable, we’re equipped to integrate seamlessly with your system. This flexibility ensures that all necessary A/B testing information is organized and accessible, fostering an efficient workflow.
Step 6: Consistent Updates and Open Communication
Regular catch-ups and a dedicated communication channel, like Teams, are central to our approach. These ensure continuous, effective communication, vital for monitoring project progress and adapting to evolving needs.
Your A/B Testing, Our Expertise
Client onboarding at EchoLogyx is more than a process; it’s the first chapter in a story of A/B testing triumph. With our commitment to personalized service and adaptive strategies, we’re ready to bring your A/B testing projects to life.
Eager to embark on this journey with us? Click the link in the comments to contact us and schedule a call! We’re excited to collaborate and achieve great results together.
At EchoLogyx, we’re always on the lookout for tools that streamline our coding process, enhance efficiency, and foster collaboration. This is why we’re excited about integrating GitHub Copilot into our development workflow. Let’s delve into how this AI-powered tool is transforming the way we code.
Speedy Coding with Autocomplete
GitHub Copilot’s autocomplete feature is a game changer. It suggests code as you type, significantly speeding up the development process by reducing the need to type out full lines of code. This is particularly useful when dealing with repetitive code patterns.
Intelligent Code Generation
Based on context and user input, GitHub Copilot can generate entire code snippets. This capability is invaluable in saving time and effort, especially when creating boilerplate code that forms the backbone of many A/B Testing Variations.
Multilingual Support
With support for multiple programming languages, we are leveraging Copilot’s features when it comes to Full Stack test development. This flexibility ensures that our team can work efficiently regardless of the language used – whether we need to develop using JS or Liquid.
Context-Aware Suggestions
Copilot is not just about generating code; it’s about generating relevant code. By analyzing the current codebase and project context, it provides suggestions that are accurate and fitting to the specific development scenario.
Advanced Code Completion
Particularly impressive is Copilot’s ability to complete code snippets based on partial inputs. This reduces manual typing and significantly enhances coding speed, which is especially advantageous for projects with strict time constraints.
Error Detection and Resolution
Error detection is another area where Copilot excels. It can identify common coding errors and suggest fixes, improving overall code quality and reducing debugging time.
Efficient Code Refactoring
Refactoring code for better readability and maintainability is crucial. Copilot suggests refactoring options, streamlining what would otherwise be a time-consuming manual process.
Automated Code Documentation
Adequate documentation is vital but often overlooked. Copilot can generate documentation and comments based on the code context, easing the burden of manual documentation.
Code Optimization Guidance
GitHub Copilot doesn’t just write code; it suggests optimizations and best practices to improve code performance and efficiency. This ensures that our A/B testing variation code is not only functional but also high-performing.
Collaborative Code Development
Perhaps one of its most impactful features is its ability to aid in code collaboration. By providing suggestions and insights based on shared code repositories, Copilot enhances our team collaboration efforts.
Adopting GitHub Copilot is more than just embracing a tool; it’s about embracing a future where AI partners with human creativity to produce outstanding software solutions. As we integrate Copilot into our workflow, we’re excited to see how it will continue to refine our processes and products.
One thing as a caveat that we would like to put out there – Co-pilot is NOT replacing the developers. It is acting as a helping hand. Based on our experience with developing and pushing out around 500 A/B tests each month, using Co-Pilot is saving between 5% to 10% of the variation development time. Even when it writes the code based on the given context, our engineers then need to go through and update based on the website/pages and the A/B testing variation including supporting the different use cases.
At EchoLogyx, we’re always on the lookout for tools that streamline our coding process, enhance efficiency, and foster collaboration. This is why we’re excited about integrating GitHub Copilot into our development workflow. Let’s delve into how this AI-powered tool is transforming the way we code.
Speedy Coding with Autocomplete
GitHub Copilot’s autocomplete feature is a game changer. It suggests code as you type, significantly speeding up the development process by reducing the need to type out full lines of code. This is particularly useful when dealing with repetitive code patterns.
Intelligent Code Generation
Based on context and user input, GitHub Copilot can generate entire code snippets. This capability is invaluable in saving time and effort, especially when creating boilerplate code that forms the backbone of many A/B Testing Variations.
Multilingual Support
With support for multiple programming languages, we are leveraging Copilot’s features when it comes to Full Stack test development. This flexibility ensures that our team can work efficiently regardless of the language used – whether we need to develop using JS or Liquid.
Context-Aware Suggestions
Copilot is not just about generating code; it’s about generating relevant code. By analyzing the current codebase and project context, it provides suggestions that are accurate and fitting to the specific development scenario.
Advanced Code Completion
Particularly impressive is Copilot’s ability to complete code snippets based on partial inputs. This reduces manual typing and significantly enhances coding speed, which is especially advantageous for projects with strict time constraints.
Error Detection and Resolution
Error detection is another area where Copilot excels. It can identify common coding errors and suggest fixes, improving overall code quality and reducing debugging time.
Efficient Code Refactoring
Refactoring code for better readability and maintainability is crucial. Copilot suggests refactoring options, streamlining what would otherwise be a time-consuming manual process.
Automated Code Documentation
Adequate documentation is vital but often overlooked. Copilot can generate documentation and comments based on the code context, easing the burden of manual documentation.
Code Optimization Guidance
GitHub Copilot doesn’t just write code; it suggests optimizations and best practices to improve code performance and efficiency. This ensures that our A/B testing variation code is not only functional but also high-performing.
Collaborative Code Development
Perhaps one of its most impactful features is its ability to aid in code collaboration. By providing suggestions and insights based on shared code repositories, Copilot enhances our team collaboration efforts.
Adopting GitHub Copilot is more than just embracing a tool; it’s about embracing a future where AI partners with human creativity to produce outstanding software solutions. As we integrate Copilot into our workflow, we’re excited to see how it will continue to refine our processes and products.
One thing as a caveat that we would like to put out there – Co-pilot is NOT replacing the developers. It is acting as a helping hand. Based on our experience with developing and pushing out around 500 A/B tests each month, using Co-Pilot is saving between 5% to 10% of the variation development time. Even when it writes the code based on the given context, our engineers then need to go through and update based on the website/pages and the A/B testing variation including supporting the different use cases.
As we navigate through 2024, the digital landscape is evolving at an unprecedented pace, making it crucial for businesses to stay ahead. Key trends in Conversion Rate Optimization (CRO) are increasingly influenced by a deep understanding of technology and user behavior.
Mobile Optimization Priority and the Impact of 5G Technology
The emergence of 5G technology has significantly amplified the importance of mobile optimization for businesses. As we step into this new era, ensuring that websites are fully optimized for mobile devices is more crucial than ever. This involves enhancing website responsiveness, loading speeds, and overall user experience on mobile platforms. Embracing the advancements brought by 5G technology, businesses can leverage various strategies and best practices to ensure their websites perform optimally on mobile devices, thereby improving user engagement and satisfaction.
Building Trust Online with Authentic Customer Testimonials
The digital age has seen challenges in building trust, particularly due to fake reviews. Authentic customer testimonials have become increasingly vital. A Harvard Business School study revealed that nearly 20% of Yelp reviews are fraudulent, impacting consumer trust and brand reputation significantly. Businesses are now adopting innovative methods like transaction-linked reviews and AI for detecting fake reviews, highlighting the importance of authenticity in the digital space.
Website Speed Enhancement
Website speed is critical for user retention and SEO rankings. Tools like Google PageSpeed Insights are invaluable in providing insights into site performance and optimization strategies. Enhancing website speed through image optimization, content delivery networks (CDNs), and minifying CSS, JavaScript, and HTML not only improves user experience but also positively impacts search engine rankings.
The Critical Role of Site Speed in CRO
The importance of site speed in CRO is underscored by several studies:
Portent’s Findings: A study by Portent found a direct correlation between page load times and conversion rates. Pages that loaded in 2.4 seconds had a 1.9% conversion rate, which dropped to 1.5% at 3.3 seconds and below 1% at 4.2 seconds. The rate plummeted to just 0.6% for load times of 5.7 seconds or more.
HubSpot Insights: According to HubSpot, the average website load time in 2023 was approximately 2.5 seconds on desktop and 8.6 seconds on mobile. Optimal conversion rates were observed with load times between 0-4 seconds. Notably, a B2B site loading in 1 second had a conversion rate 3x higher than one taking 5 seconds to load.
Cloudflare’s Analysis: Supporting these findings, Cloudflare’s research emphasizes the substantial effect of website performance on conversion rates. For example, Walmart observed a 2% increase in conversions for every 1-second improvement in page load time.
Seamless Cross-Device Experience
With users frequently switching between devices, ensuring a seamless experience across all platforms is crucial. Responsive and adaptive web designs are key to maintaining a consistent look and feel across devices, thereby enhancing user engagement and reducing bounce rates.
The 2024 CRO trends in mobile optimization, building online trust, enhancing website speed, and ensuring a seamless cross-device experience reflect an evolved understanding of user expectations and technological advancements. Businesses that adapt to these trends are well-positioned to succeed in the digital marketplace. Prioritizing faster load times, optimizing user experience across devices, and maintaining authenticity in customer interactions are essential strategies in this rapidly evolving digital landscape.
As we navigate through 2024, the digital landscape is evolving at an unprecedented pace, making it crucial for businesses to stay ahead. Key trends in Conversion Rate Optimization (CRO) are increasingly influenced by a deep understanding of technology and user behavior.
Mobile Optimization Priority and the Impact of 5G Technology
The emergence of 5G technology has significantly amplified the importance of mobile optimization for businesses. As we step into this new era, ensuring that websites are fully optimized for mobile devices is more crucial than ever. This involves enhancing website responsiveness, loading speeds, and overall user experience on mobile platforms. Embracing the advancements brought by 5G technology, businesses can leverage various strategies and best practices to ensure their websites perform optimally on mobile devices, thereby improving user engagement and satisfaction.
Building Trust Online with Authentic Customer Testimonials
The digital age has seen challenges in building trust, particularly due to fake reviews. Authentic customer testimonials have become increasingly vital. A Harvard Business School study revealed that nearly 20% of Yelp reviews are fraudulent, impacting consumer trust and brand reputation significantly. Businesses are now adopting innovative methods like transaction-linked reviews and AI for detecting fake reviews, highlighting the importance of authenticity in the digital space.
Website Speed Enhancement
Website speed is critical for user retention and SEO rankings. Tools like Google PageSpeed Insights are invaluable in providing insights into site performance and optimization strategies. Enhancing website speed through image optimization, content delivery networks (CDNs), and minifying CSS, JavaScript, and HTML not only improves user experience but also positively impacts search engine rankings.
The Critical Role of Site Speed in CRO
The importance of site speed in CRO is underscored by several studies:
Portent’s Findings: A study by Portent found a direct correlation between page load times and conversion rates. Pages that loaded in 2.4 seconds had a 1.9% conversion rate, which dropped to 1.5% at 3.3 seconds and below 1% at 4.2 seconds. The rate plummeted to just 0.6% for load times of 5.7 seconds or more.
HubSpot Insights: According to HubSpot, the average website load time in 2023 was approximately 2.5 seconds on desktop and 8.6 seconds on mobile. Optimal conversion rates were observed with load times between 0-4 seconds. Notably, a B2B site loading in 1 second had a conversion rate 3x higher than one taking 5 seconds to load.
Cloudflare’s Analysis: Supporting these findings, Cloudflare’s research emphasizes the substantial effect of website performance on conversion rates. For example, Walmart observed a 2% increase in conversions for every 1-second improvement in page load time.
Seamless Cross-Device Experience
With users frequently switching between devices, ensuring a seamless experience across all platforms is crucial. Responsive and adaptive web designs are key to maintaining a consistent look and feel across devices, thereby enhancing user engagement and reducing bounce rates.
The 2024 CRO trends in mobile optimization, building online trust, enhancing website speed, and ensuring a seamless cross-device experience reflect an evolved understanding of user expectations and technological advancements. Businesses that adapt to these trends are well-positioned to succeed in the digital marketplace. Prioritizing faster load times, optimizing user experience across devices, and maintaining authenticity in customer interactions are essential strategies in this rapidly evolving digital landscape.
EchoLogyx had the distinct pleasure of attending the recent Experimentation Elite Conference in London, a hallmark event in the realm of Conversion Rate Optimization (CRO) and digital experimentation. The conference was a vibrant melting pot of innovative strategies and cutting-edge ideas, hosting an array of experts and thought leaders from the world of CRO and experimentation. It was an enlightening experience, offering us rich insights to further refine our strategies in key areas like CRO and A/B testing.
Jim Sterne – The AI Visionary
Unveiling Generative AI: Jim Sterne’s engaging presentation took us through the evolution of AI and provided an in-depth understanding of Generative AI tools like ChatGPT and Midjourney. His emphasis on the practical application of AI in our daily work was particularly insightful, reinforcing the idea that AI is a tool to be leveraged for efficiency and innovation.
Futuristic Perspectives: Sterne’s vision of the future, where personal AIs streamline our lives using our own data, opened up a new realm of possibilities. His talk was a reminder that the future of AI is closer than we think, and it’s imperative to adapt and evolve with these technological advancements.
Bhavik Patel – The CRO Historian
From Past to Future: Bhavik Patel’s talk was a journey through the history of CRO, from early clinical trials to the latest developments in the field. His comprehensive overview highlighted the importance of understanding the roots of online experimentation and using this knowledge to innovate and adapt in the present.
A Vision for AI in CRO: Patel’s focus on AI’s role in shaping the future of CRO aligned perfectly with our focus at EchoLogyx. His insights into how AI can enhance CRO processes and strategies resonated with our commitment to staying at the forefront of digital innovation.
Jonny Longden – The Strategic Optimizer
Strategic Experimentation: Jonny Longden’s session on making experimentation more strategic was a deep dive into the development of experimentation and CRO teams. His approach to redefining ‘Conversion Rate Optimization’ and emphasizing learning over mere optimization was enlightening.
The Four-Step Plan: Longden’s four-step plan for a more strategic approach to experimentation—focusing on customer strategy, critical thinking, breaking silos, and broadening validation concepts—was a valuable takeaway for us, offering a structured path to refining our experimentation strategies.
Lucas Vos – The Myth Buster
Busting Experimentation Myths: Lucas Vos’s presentation was an eye-opener on the myths that can hinder an effective experimentation program. His emphasis on understanding stakeholder concerns and addressing them effectively is something we, at EchoLogyx, strongly resonate with.
Effective Stakeholder Engagement: Vos’s strategies for aligning experimentation efforts with stakeholder concerns and demonstrating the value of experimentation in decision-making were particularly valuable. His insights will undoubtedly aid us in enhancing our stakeholder engagement practices.
Mirjam De Klepper – The Ethical Advocate
Ethics in CRO: Mirjam De Klepper’s talk on the ethics of CRO was a refreshing perspective. Her emphasis on the responsibility that comes with the power to influence through experimentation underscored the importance of ethical considerations in our work.
Ben Labay – The ROI Expert
Experimentation ROI: Ben Labay’s insights into measuring the ROI of experimentation programs, particularly his demonstration of the ‘flicker effect’, provided us with new perspectives on evaluating the impact of our strategies.
Annemarie Klaassen – The CRO Product Manager
Treating CRO as a Product: Annemarie Klaassen’s approach to treating CRO programs as products was a novel concept that we found intriguing. Her emphasis
on transparency and stakeholder engagement aligns with our ethos at EchoLogyx.
Daphne Tideman – The Proponent of Learning from Failure
Learning from Failures: Daphne Tideman’s focus on turning experimentation failures into learning opportunities was a highlight. Her approach to strategic planning in experimentation aligns well with our continuous improvement philosophy at EchoLogyx.
Michael Aagaard – The Disappointment Psychologist
Understanding Customer Expectations: Michael Aagaard’s closing talk on the science and psychology of disappointment was a powerful reminder of the importance of understanding and managing customer expectations, a principle that is central to our work at EchoLogyx.
Humorous Key Takeaways: As enlightening as the conference was, it wasn’t all serious business. Here are some humorous highlights that brought smiles and laughter to the attendees:
Michael Aagaard: The Sea Monkey Disappointment
Michael Aagaard’s childhood experience with sea monkeys highlights the gap between marketing hype and reality. He expected an aquatic marvel but faced disappointment, showcasing how exaggerated marketing can lead to customer dissatisfaction. This story is a classic example of the consequences of unmet expectations in customer experiences.
Annemarie Klaassen: Embracing Failures
Annemarie Klaassen’s concept of celebrating failures annually offers a fresh perspective on setbacks. Instead of viewing failures negatively, she suggests embracing them as learning opportunities, in line with EchoLogyx’s philosophy of using challenges for growth. This approach transforms our understanding of failures into moments of insight.
Daphne Tideman: Journey to Half Iron Woman
Daphne Tideman’s journey from a bike accident to completing the Half Iron Man illustrates resilience. Her story, mirroring EchoLogyx’s ethos, shows that setbacks are not the end but beginnings of success paths, emphasizing the importance of persistence and learning from challenges.
The London Experimentation Elite Conference was not just an event, but a journey of profound learning and inspiration. It reinforced our commitment to staying at the cutting edge of CRO and digital experimentation strategies. A heartfelt appreciation to all the speakers for their invaluable insights and to the organizers for creating such a stimulating and enriching environment. We eagerly anticipate applying these learnings to our work, further enhancing our strategies and methodologies in the ever-evolving digital landscape.
EchoLogyx had the distinct pleasure of attending the recent Experimentation Elite Conference in London, a hallmark event in the realm of Conversion Rate Optimization (CRO) and digital experimentation. The conference was a vibrant melting pot of innovative strategies and cutting-edge ideas, hosting an array of experts and thought leaders from the world of CRO and experimentation. It was an enlightening experience, offering us rich insights to further refine our strategies in key areas like CRO and A/B testing.
Jim Sterne – The AI Visionary
Unveiling Generative AI: Jim Sterne’s engaging presentation took us through the evolution of AI and provided an in-depth understanding of Generative AI tools like ChatGPT and Midjourney. His emphasis on the practical application of AI in our daily work was particularly insightful, reinforcing the idea that AI is a tool to be leveraged for efficiency and innovation.
Futuristic Perspectives: Sterne’s vision of the future, where personal AIs streamline our lives using our own data, opened up a new realm of possibilities. His talk was a reminder that the future of AI is closer than we think, and it’s imperative to adapt and evolve with these technological advancements.
Bhavik Patel – The CRO Historian
From Past to Future: Bhavik Patel’s talk was a journey through the history of CRO, from early clinical trials to the latest developments in the field. His comprehensive overview highlighted the importance of understanding the roots of online experimentation and using this knowledge to innovate and adapt in the present.
A Vision for AI in CRO: Patel’s focus on AI’s role in shaping the future of CRO aligned perfectly with our focus at EchoLogyx. His insights into how AI can enhance CRO processes and strategies resonated with our commitment to staying at the forefront of digital innovation.
Jonny Longden – The Strategic Optimizer
Strategic Experimentation: Jonny Longden’s session on making experimentation more strategic was a deep dive into the development of experimentation and CRO teams. His approach to redefining ‘Conversion Rate Optimization’ and emphasizing learning over mere optimization was enlightening.
The Four-Step Plan: Longden’s four-step plan for a more strategic approach to experimentation—focusing on customer strategy, critical thinking, breaking silos, and broadening validation concepts—was a valuable takeaway for us, offering a structured path to refining our experimentation strategies.
Lucas Vos – The Myth Buster
Busting Experimentation Myths: Lucas Vos’s presentation was an eye-opener on the myths that can hinder an effective experimentation program. His emphasis on understanding stakeholder concerns and addressing them effectively is something we, at EchoLogyx, strongly resonate with.
Effective Stakeholder Engagement: Vos’s strategies for aligning experimentation efforts with stakeholder concerns and demonstrating the value of experimentation in decision-making were particularly valuable. His insights will undoubtedly aid us in enhancing our stakeholder engagement practices.
Mirjam De Klepper – The Ethical Advocate
Ethics in CRO: Mirjam De Klepper’s talk on the ethics of CRO was a refreshing perspective. Her emphasis on the responsibility that comes with the power to influence through experimentation underscored the importance of ethical considerations in our work.
Ben Labay – The ROI Expert
Experimentation ROI: Ben Labay’s insights into measuring the ROI of experimentation programs, particularly his demonstration of the ‘flicker effect’, provided us with new perspectives on evaluating the impact of our strategies.
Annemarie Klaassen – The CRO Product Manager
Treating CRO as a Product: Annemarie Klaassen’s approach to treating CRO programs as products was a novel concept that we found intriguing. Her emphasis
on transparency and stakeholder engagement aligns with our ethos at EchoLogyx.
Daphne Tideman – The Proponent of Learning from Failure
Learning from Failures: Daphne Tideman’s focus on turning experimentation failures into learning opportunities was a highlight. Her approach to strategic planning in experimentation aligns well with our continuous improvement philosophy at EchoLogyx.
Michael Aagaard – The Disappointment Psychologist
Understanding Customer Expectations: Michael Aagaard’s closing talk on the science and psychology of disappointment was a powerful reminder of the importance of understanding and managing customer expectations, a principle that is central to our work at EchoLogyx.
Humorous Key Takeaways: As enlightening as the conference was, it wasn’t all serious business. Here are some humorous highlights that brought smiles and laughter to the attendees:
Michael Aagaard: The Sea Monkey Disappointment
Michael Aagaard’s childhood experience with sea monkeys highlights the gap between marketing hype and reality. He expected an aquatic marvel but faced disappointment, showcasing how exaggerated marketing can lead to customer dissatisfaction. This story is a classic example of the consequences of unmet expectations in customer experiences.
Annemarie Klaassen: Embracing Failures
Annemarie Klaassen’s concept of celebrating failures annually offers a fresh perspective on setbacks. Instead of viewing failures negatively, she suggests embracing them as learning opportunities, in line with EchoLogyx’s philosophy of using challenges for growth. This approach transforms our understanding of failures into moments of insight.
Daphne Tideman: Journey to Half Iron Woman
Daphne Tideman’s journey from a bike accident to completing the Half Iron Man illustrates resilience. Her story, mirroring EchoLogyx’s ethos, shows that setbacks are not the end but beginnings of success paths, emphasizing the importance of persistence and learning from challenges.
The London Experimentation Elite Conference was not just an event, but a journey of profound learning and inspiration. It reinforced our commitment to staying at the cutting edge of CRO and digital experimentation strategies. A heartfelt appreciation to all the speakers for their invaluable insights and to the organizers for creating such a stimulating and enriching environment. We eagerly anticipate applying these learnings to our work, further enhancing our strategies and methodologies in the ever-evolving digital landscape.
Embracing the Festive Spirit in Marketing
Season’s greetings! The holiday season has arrived, bringing with it a flurry of opportunities for businesses to connect with their audience. In this festive period, the magic of Christmas campaigns comes to life, and what better way to enhance this magic than with A/B testing? This technique acts like a helpful elf, meticulously refining your messages to ensure they resonate joyfully with your audience.
The Joyful Journey of A/B Testing in Holiday Campaigns
Think of A/B testing as a delightful holiday tradition, akin to choosing between gingerbread and sugar cookies. It involves comparing two versions of your marketing message to discover which one is most cherished by your audience. This approach is your sleigh ride over the competitive landscape during this season of cheer and excitement.
Exploring the Festive Facets of A/B Testing in Christmas Campaigns
Crafting Emails That Resonate with Holiday Joy: The challenge is to create email subject lines and content that evoke the warmth and excitement of Christmas morning. It’s about finding the right words, whether it’s “Santa’s Got a Sale!” or “Unwrap Your Holiday Discount!”, to make your customers feel the joy and anticipation of the season.
Designing Visuals That Sparkle with Festivity: The visual appeal of your campaign can range from the classic charm of ‘Miracle on 34th Street’ to the playful whimsy of ‘Elf’. Testing different festive themes and layouts helps determine what truly captivates and delights your audience, making them feel as if they’re dancing in a winter wonderland.
Offering Promotions That Light Up the Season: The holiday season is a time for special offers, whether it’s discounts, free shipping, or other festive treats. The key is to test various promotions to discover what ignites the holiday spirit in your customers, making their season bright and merry .
Perfecting the Timing and Frequency of Your Messages: In the world of holiday marketing, timing can be as crucial as the perfect gift. It’s important to test and find the ideal moments to send your messages, ensuring they are received as warmly as a heartfelt holiday greeting, rather than an untimely interruption .
Delivery Cut-Off Period: Test messaging around the last order dates for guaranteed holiday delivery, potentially using countdown timers for urgency.
Gift Packaging Options: Include testing for the appeal of special gift packaging, gift cards, and personalized messages to enhance the gifting experience.
Best Practices for a Successful A/B Testing Season
Embarking Early on the Testing Journey: Just like the early preparations for the holiday season, starting your A/B testing early is key to staying ahead and ensuring your campaigns are as polished as the star atop your tree .
Seeking Clarity and Significance in Results: Your testing outcomes should be as clear and meaningful as the holiday lights that illuminate the winter nights .
Focusing on One Element at a Time: To avoid confusion and ensure accuracy, it’s important to test one variable at a time, much like carefully placing each ornament on the tree .
Adapting Based on Insights Gained: Use the valuable insights from your A/B testing to continuously refine and brighten your campaign, just as you would adjust your holiday decorations for the perfect festive ambiance .
The Gift of Insightful Marketing A/B testing is akin to a secret Santa in the world of marketing, offering the gift of insight that helps unwrap the full potential of your Christmas campaigns. It’s a strategy that leads to jolly results and merry returns
Embrace the Season of Testing So, put on your festive marketing hat and dive into the world of A/B testing. Let’s spread holiday cheer and insightful strategies for a prosperous and joyful new year!
Season’s greetings! The holiday season has arrived, bringing with it a flurry of opportunities for businesses to connect with their audience. In this festive period, the magic of Christmas campaigns comes to life, and what better way to enhance this magic than with A/B testing? This technique acts like a helpful elf, meticulously refining your messages to ensure they resonate joyfully with your audience.
The Joyful Journey of A/B Testing in Holiday Campaigns
Think of A/B testing as a delightful holiday tradition, akin to choosing between gingerbread and sugar cookies. It involves comparing two versions of your marketing message to discover which one is most cherished by your audience. This approach is your sleigh ride over the competitive landscape during this season of cheer and excitement.
Exploring the Festive Facets of A/B Testing in Christmas Campaigns
Crafting Emails That Resonate with Holiday Joy: The challenge is to create email subject lines and content that evoke the warmth and excitement of Christmas morning. It’s about finding the right words, whether it’s “Santa’s Got a Sale!” or “Unwrap Your Holiday Discount!”, to make your customers feel the joy and anticipation of the season.
Designing Visuals That Sparkle with Festivity: The visual appeal of your campaign can range from the classic charm of ‘Miracle on 34th Street’ to the playful whimsy of ‘Elf’. Testing different festive themes and layouts helps determine what truly captivates and delights your audience, making them feel as if they’re dancing in a winter wonderland.
Offering Promotions That Light Up the Season: The holiday season is a time for special offers, whether it’s discounts, free shipping, or other festive treats. The key is to test various promotions to discover what ignites the holiday spirit in your customers, making their season bright and merry .
Perfecting the Timing and Frequency of Your Messages: In the world of holiday marketing, timing can be as crucial as the perfect gift. It’s important to test and find the ideal moments to send your messages, ensuring they are received as warmly as a heartfelt holiday greeting, rather than an untimely interruption .
Delivery Cut-Off Period: Test messaging around the last order dates for guaranteed holiday delivery, potentially using countdown timers for urgency.
Gift Packaging Options: Include testing for the appeal of special gift packaging, gift cards, and personalized messages to enhance the gifting experience.
Best Practices for a Successful A/B Testing Season
Embarking Early on the Testing Journey: Just like the early preparations for the holiday season, starting your A/B testing early is key to staying ahead and ensuring your campaigns are as polished as the star atop your tree .
Seeking Clarity and Significance in Results: Your testing outcomes should be as clear and meaningful as the holiday lights that illuminate the winter nights .
Focusing on One Element at a Time: To avoid confusion and ensure accuracy, it’s important to test one variable at a time, much like carefully placing each ornament on the tree .
Adapting Based on Insights Gained: Use the valuable insights from your A/B testing to continuously refine and brighten your campaign, just as you would adjust your holiday decorations for the perfect festive ambiance .
The Gift of Insightful Marketing A/B testing is akin to a secret Santa in the world of marketing, offering the gift of insight that helps unwrap the full potential of your Christmas campaigns. It’s a strategy that leads to jolly results and merry returns
Embrace the Season of Testing So, put on your festive marketing hat and dive into the world of A/B testing. Let’s spread holiday cheer and insightful strategies for a prosperous and joyful new year!
The EchoLogyx team recently had the incredible opportunity to attend the Conversion Hotel Conference 2023, a premier event at the forefront of digital marketing and innovation. This conference was a melting pot of groundbreaking ideas and strategies, bringing together some of the most brilliant minds in the industry. We’re excited to delve deeper into the insights shared by each speaker, particularly those that align perfectly with our focus on Conversion Rate Optimisation (CRO), A/B testing, and enhancing digital strategies. The event provided us with a wealth of knowledge and inspiration to further refine our approaches in these key areas.
Joanna Wiebe – The Artisan of Words
Crafting Impactful Copy: Joanna Wiebe, renowned for her expertise in copywriting, emphasized the power of words in customer engagement. She illustrated how meticulously chosen words could significantly impact customer behavior and conversion rates. Her session was a masterclass in crafting copy that not only captures attention but also resonates on a deeper level with the audience.
Customer-Centric Writing: Joanna’s approach to copywriting is deeply rooted in understanding the customer’s perspective. She stressed the importance of empathy in writing, encouraging us to step into our customers’ shoes to create more meaningful and impactful messages.
Else Aerts – The Research Connoisseur
Pursuit of Authenticity: Else Aerts brought a critical eye to the importance of conducting thorough and unbiased research. She underscored the necessity of authenticity in data collection and analysis, highlighting how genuine insights are pivotal in shaping effective strategies.
Overcoming Research Biases: Else also tackled the often-overlooked issue of biases in research. She provided practical strategies to identify and avoid these biases, ensuring that the research remains objective and reliable.
Greg Shapiro – The Cultural Humorist
Leveraging Humor: Greg Shapiro’s session was a refreshing take on using humor in marketing. He demonstrated how effectively humor could bridge communication gaps, making messages more memorable and engaging.
Humor in a Global Context: Greg also touched upon the nuances of cross-cultural humor. His insights into adapting humor to fit different cultural contexts were particularly enlightening, offering a new perspective on global marketing strategies.
Tim Stewart – The Strategic Visionary
Unveiling ‘Selling Magic Beans’: In his presentation ‘Selling Magic Beans’, Tim Stewart provided a comprehensive look at the intersection of art, science, and sales in the digital marketing landscape. He emphasized the importance of skillful communication and understanding in various professional roles.
Embracing the PCMC Method: Stewart’s introduction of the ‘PCMC’ (Probe, Confirm, Match, Close) framework shed light on a strategic approach to business communication. He showed how these principles, integral to digital marketing, are equally important in enhancing personal and professional interactions and negotiations.
Lucia van den Brink – The Teamwork Advocate
Power of Collaboration: Lucia van den Brink’s session was a testament to the power of teamwork in innovation. She highlighted how collaborative efforts could lead to more creative solutions and drive success in projects.
Goal Congruence: Lucia also emphasized the importance of aligning team goals with organizational objectives. This alignment, she argued, is crucial for ensuring that everyone is working towards a common purpose.
André Morys – The Digital Innovator
Embracing Data Wisdom: André taught us the significance of data in shaping digital landscapes.
“You shall not pass”: Get around the Hippo problem – how to navigate through the team to get the buy in.
Simone Neeling – The Advocate of Resilience
Navigating Long COVID: Simone Neeling shared her personal journey with long COVID, offering insights and strategies for resilience. Her story was not only inspiring but also provided practical advice for those facing similar challenges.
Lifestyle Adaptation: Simone’s session was also a deep dive into adapting lifestyles to new challenges. Her tips were grounded in reality and offered a roadmap for thriving in the face of adversity.
Karl Gilis – The UX Evangelist
Fundamental UX Principles: Karl Gilis reaffirmed the importance of sticking to fundamental UX principles. His emphasis on user-centric design principles was a reminder of the core values that drive effective UX design.
User-Centric Design: Karl’s advocacy for a design approach that prioritizes user feedback highlighted the importance of listening to and understanding the user’s needs and preferences.
Annegien Bruins Slot – The SEO-CRO Strategist
SEO-CRO Symbiosis: Annegien Bruins Slot’s expertise in both SEO and CRO was evident as she discussed the integration of these two critical elements of digital marketing. She provided insights into how combining SEO and CRO tactics can lead to a more effective overall strategy.
Strategic Integration: Annegien’s approach to integrating SEO and CRO was akin to conducting a symphony – each element working in harmony to create a successful campaign.
Panels and unconference sessions
Great panels hosted by Charles Meaden talking about AI in CRO, Ethical aspect, Statistic in CRO, User Journey mapping. There were quite a few unconference sessions ran by attendees – including SEO and CRO how and why they should be working together for the success of the business.
An Enlightening Experience for Echologyx
The Conversion Hotel Conference 2023 was not just an event but a journey of learning and growth. We at Echologyx are grateful for the opportunity to absorb such invaluable insights from industry leaders. These learnings will undoubtedly shape our strategies and approaches in the days to come, particularly in optimizing conversion rates, refining our A/B testing methodologies, and enhancing our overall digital strategies. A heartfelt thank you to all the speakers for their invaluable contributions, and to Charles Meaden for adding a unique flavor to the event, enriching our understanding and application of these critical aspects of digital marketing.
The EchoLogyx team recently had the incredible opportunity to attend the Conversion Hotel Conference 2023, a premier event at the forefront of digital marketing and innovation. This conference was a melting pot of groundbreaking ideas and strategies, bringing together some of the most brilliant minds in the industry. We’re excited to delve deeper into the insights shared by each speaker, particularly those that align perfectly with our focus on Conversion Rate Optimisation (CRO), A/B testing, and enhancing digital strategies. The event provided us with a wealth of knowledge and inspiration to further refine our approaches in these key areas.
Joanna Wiebe – The Artisan of Words
Crafting Impactful Copy: Joanna Wiebe, renowned for her expertise in copywriting, emphasized the power of words in customer engagement. She illustrated how meticulously chosen words could significantly impact customer behavior and conversion rates. Her session was a masterclass in crafting copy that not only captures attention but also resonates on a deeper level with the audience.
Customer-Centric Writing: Joanna’s approach to copywriting is deeply rooted in understanding the customer’s perspective. She stressed the importance of empathy in writing, encouraging us to step into our customers’ shoes to create more meaningful and impactful messages.
Else Aerts – The Research Connoisseur
Pursuit of Authenticity: Else Aerts brought a critical eye to the importance of conducting thorough and unbiased research. She underscored the necessity of authenticity in data collection and analysis, highlighting how genuine insights are pivotal in shaping effective strategies.
Overcoming Research Biases: Else also tackled the often-overlooked issue of biases in research. She provided practical strategies to identify and avoid these biases, ensuring that the research remains objective and reliable.
Greg Shapiro – The Cultural Humorist
Leveraging Humor: Greg Shapiro’s session was a refreshing take on using humor in marketing. He demonstrated how effectively humor could bridge communication gaps, making messages more memorable and engaging.
Humor in a Global Context: Greg also touched upon the nuances of cross-cultural humor. His insights into adapting humor to fit different cultural contexts were particularly enlightening, offering a new perspective on global marketing strategies.
Tim Stewart – The Strategic Visionary
Unveiling ‘Selling Magic Beans’: In his presentation ‘Selling Magic Beans’, Tim Stewart provided a comprehensive look at the intersection of art, science, and sales in the digital marketing landscape. He emphasized the importance of skillful communication and understanding in various professional roles.
Embracing the PCMC Method: Stewart’s introduction of the ‘PCMC’ (Probe, Confirm, Match, Close) framework shed light on a strategic approach to business communication. He showed how these principles, integral to digital marketing, are equally important in enhancing personal and professional interactions and negotiations.
Lucia van den Brink – The Teamwork Advocate
Power of Collaboration: Lucia van den Brink’s session was a testament to the power of teamwork in innovation. She highlighted how collaborative efforts could lead to more creative solutions and drive success in projects.
Goal Congruence: Lucia also emphasized the importance of aligning team goals with organizational objectives. This alignment, she argued, is crucial for ensuring that everyone is working towards a common purpose.
André Morys – The Digital Innovator
Embracing Data Wisdom: André taught us the significance of data in shaping digital landscapes.
“You shall not pass”: Get around the Hippo problem – how to navigate through the team to get the buy in.
Simone Neeling – The Advocate of Resilience
Navigating Long COVID: Simone Neeling shared her personal journey with long COVID, offering insights and strategies for resilience. Her story was not only inspiring but also provided practical advice for those facing similar challenges.
Lifestyle Adaptation: Simone’s session was also a deep dive into adapting lifestyles to new challenges. Her tips were grounded in reality and offered a roadmap for thriving in the face of adversity.
Karl Gilis – The UX Evangelist
Fundamental UX Principles: Karl Gilis reaffirmed the importance of sticking to fundamental UX principles. His emphasis on user-centric design principles was a reminder of the core values that drive effective UX design.
User-Centric Design: Karl’s advocacy for a design approach that prioritizes user feedback highlighted the importance of listening to and understanding the user’s needs and preferences.
Annegien Bruins Slot – The SEO-CRO Strategist
SEO-CRO Symbiosis: Annegien Bruins Slot’s expertise in both SEO and CRO was evident as she discussed the integration of these two critical elements of digital marketing. She provided insights into how combining SEO and CRO tactics can lead to a more effective overall strategy.
Strategic Integration: Annegien’s approach to integrating SEO and CRO was akin to conducting a symphony – each element working in harmony to create a successful campaign.
Panels and unconference sessions
Great panels hosted by Charles Meaden talking about AI in CRO, Ethical aspect, Statistic in CRO, User Journey mapping. There were quite a few unconference sessions ran by attendees – including SEO and CRO how and why they should be working together for the success of the business.
An Enlightening Experience for Echologyx
The Conversion Hotel Conference 2023 was not just an event but a journey of learning and growth. We at Echologyx are grateful for the opportunity to absorb such invaluable insights from industry leaders. These learnings will undoubtedly shape our strategies and approaches in the days to come, particularly in optimizing conversion rates, refining our A/B testing methodologies, and enhancing our overall digital strategies. A heartfelt thank you to all the speakers for their invaluable contributions, and to Charles Meaden for adding a unique flavor to the event, enriching our understanding and application of these critical aspects of digital marketing.
Having a clear test plan is a MUST for any A/B Tests.
There is no shortcut.
There is no assumption to be made.
To develop a test without the full test plan is like getting a flatpack furniture delivered without clear instructions to build it!
If you spend a few minutes extra to make sure that the test plan and the design of the variations are clarified, you will save hours of back and forth related to the test. Trust me – I have been there!
It does not matter who is developing and QAing the test, the Test plan needs to make sure that everything is covered. Whether they are targeting related, metrics related or handling different scenarios.
Use a template for the test plan if necessary that will always make sure that you are filling the required sections for your experiments.
Here is a test plan template – feel free to use it!
Having a clear test plan is a MUST for any A/B Tests.
There is no shortcut.
There is no assumption to be made.
To develop a test without the full test plan is like getting a flatpack furniture delivered without clear instructions to build it!
If you spend a few minutes extra to make sure that the test plan and the design of the variations are clarified, you will save hours of back and forth related to the test. Trust me – I have been there!
It does not matter who is developing and QAing the test, the Test plan needs to make sure that everything is covered. Whether they are targeting related, metrics related or handling different scenarios.
Use a template for the test plan if necessary that will always make sure that you are filling the required sections for your experiments.
Here is a test plan template – feel free to use it!
Want to check the metrics, goals, events for your experiments? Not sure if they are firing properly? Tired of going through the network tab and find the right calls?
EchoLogyx Test Metrics Debugger allows you to check the goals and metrics – if they are firing at the right time, and with the right interaction. Simply, add the extension, enable it, visit the page where the test is live and see whether the metrics are firing or not.
You can see the detailed information about the metrics that are being fired and what information is being passed to the testing tool.
Currently, this extension works with:
Convert
VWO
Optimizely
Adobe Target (mbox V1 and mbox V2)
Dynamic Yield
AB Tasty
Google Optimize / Analytics
We are working to cover more A/B testing tools to easily check metrics and goals.
Want to check the metrics, goals, events for your experiments? Not sure if they are firing properly? Tired of going through the network tab and find the right calls?
EchoLogyx Test Metrics Debugger allows you to check the goals and metrics – if they are firing at the right time, and with the right interaction. Simply, add the extension, enable it, visit the page where the test is live and see whether the metrics are firing or not.
You can see the detailed information about the metrics that are being fired and what information is being passed to the testing tool.
Currently, this extension works with:
Convert
VWO
Optimizely
Adobe Target (mbox V1 and mbox V2)
Dynamic Yield
AB Tasty
Google Optimize / Analytics
We are working to cover more A/B testing tools to easily check metrics and goals.
In this webinar, our COO Tasin Reza and the eCommerce director of eSpares.co.uk Farhan Hussain talks about their CRO program which does not only increased the efficiency but also achieved an amazing ROI of 30,000%.
Farhan talks about the process he adapted, tests he ran and also some of the winners that contributed to the overall success.
“You don’t need to always run complex tests”, says Farhan. “Sometimes really simple changes can make a massive difference to your bottom line conversion”.
Here is the Webinar replay, which was hosted by Convert.com. If you are interested to find out more about our A/B testing services or would like to see how you can achieve such ROI, simply drop us a line.
In this webinar, our COO Tasin Reza and the eCommerce director of eSpares.co.uk Farhan Hussain talks about their CRO program which does not only increased the efficiency but also achieved an amazing ROI of 30,000%.
Farhan talks about the process he adapted, tests he ran and also some of the winners that contributed to the overall success.
“You don’t need to always run complex tests”, says Farhan. “Sometimes really simple changes can make a massive difference to your bottom line conversion”.
Here is the Webinar replay, which was hosted by Convert.com. If you are interested to find out more about our A/B testing services or would like to see how you can achieve such ROI, simply drop us a line.
A simple (yet powerful) guide to AB Testing Development
Over the past decade, companies had no choice but to take the online user experience more seriously than ever before, to increase their online sales, customer loyalty and achieve their business goals.
As the online businesses race towards becoming the best amongst their competitors, CRO and AB Testing have started to play a significant role to improve digital channel performances. From large companies such as Google, Facebook, Amazon, Netflix, Microsoft to Start-ups, started experimenting with different concepts to gain additional customers or users to their website.
In this article, we are providing a simple guide to develop A/B tests. However, before doing that, it is important to understand the basics, which will help to provide a solid foundation to understand the success of CRO.
What is CRO (Conversion Rate Optimization)?
There are many definitions out there of Conversion Rate Optimization. For example, Econsultancy defines CRO as a “process of optimizing site to increase the likelihood that visitors will complete that specific action”[1] . HubSpot provides an action-focused definition of CRO by adding enabling people to take action when they visit a website. “By designing and modifying certain elements of a webpage, a business can increase the chances that site visitors will “convert” into a lead or customer before they leave”[2].
In simple terms, the definition of CRO is
“Making changes to improve metrics.”
The changes can vary from small – like a simple headline change, to large – where you could be adding a new feature to your product. The changes can be on your website, emails, mobile Apps, Search keywords, banners, or even physical entities. The ultimate thing is by making these changes, you are improving your business metrics, performance indicators or KPIs. Moreover, this process of continuous improvement of KPIs by making changes is CRO.
[1] What is conversion rate optimisation (CRO) and why do you need it? https://econsultancy.com/what-is-conversion-rate-optimisation-cro-and-why-do-you-need-it/
[2] The Beginner’s Guide to Conversion Rate Optimization (CRO) https://blog.hubspot.com/marketing/conversion-rate-optimization-guide
What is A/B Testing?
As part of the CRO program, you are coming up with ideas to make changes. These change ideas are commonly known as Hypothesis.
As with any research, a hypothesis is always validated by different forms of testing. The most common form of testing a hypothesis within the CRO program is A/B Testing (or ABn Testing) where version A signifies the control or original – what is currently live, and version B is the new variation created based on the hypothesis.
A/B Testing simply helps you to check if the changes you are making are truly improving the target KPIs. It takes the metric performance of the current version and compares it with the new version where the changes have been applied.
For example, Neil – an Optimisation Consultant has completed some research on the e-commerce site he is involved with. Based on the analytics data, he identified that on the product details pages, the price of the product is presented just below the title of the product, which some users were ignoring. His idea is to change the position of this price and make it closer to the main Call to action (CTA) button so that users can see the price before adding the item to the basket and more people will complete their purchase through this website.
This is a simple example of an AB test to find out if Neil’s hypothesis is true – by moving the price closer to the CTA, more people will complete their purchase.
Figure 1: Example of a simple AB Test where Version A or Control where the price is just below the title of the product. Version B, where the price is just above the Buy now CTA button.
A hypothesis can create more than one variation. In such cases, your AB test becomes ABn test where A still signifies the control and after that, you have variation B, variation C and so forth to signify the other variation.
In case if you are interested in changing multiple elements and test a combination of different elements, you can setup a Multivariate Test (MVT).
Multivariate Test or MVT is a form of a test where you want to find the impact of the combinations of more than one change across multiple places within a page or site section. Let us break this definition down with an example.
Let’s go back to Neil’s research, and this time, he noticed that when visitors landed on the homepage of the best retailer website, they completely ignored the Sale link and also did not notice the key USP (see below).
Figure 2: Example of a Multi-variate test where two different elements are going to be tested with multiple variations
He wants to test out two different versions of the Sale message against the control version:
Control Sale message: Up to 50% off! Shop Sale
Variation 1 Sale message: Sale now on! Get Up to 50% off
Variation 2 Sale message: Up to 50% off! Hurry while stock lasts!
At the same time, two versions of the main heading to promote free delivery and return messaging against the control:
Control Heading: Welcome to the Best Retailer site – number 1 in sustainable products
Variation 1 Heading: Number 1 Sustainable Shopping site with FREE Delivery and Returns
Variation 2 Heading: Free Delivery and Returns! Best Retailer for Sustainable products
Neil is interested in finding out not only which individual sale message or heading works best to increase the browsability and purchase, but also to see which combination of these messages work best. This is where he sets this test up as a multi-variate test or MVT to test out all of the combinations against the control.
All MVT or multi-variate tests are AB tests – with a lot of variations. For example, the total number of combinations in this example will be 9 (including the control). Each of these combinations can be considered as a variation of an AB test.
Testing tools that provide the option of setting up MVT usually gives the user some additional options – for example, excluding specific combinations and reporting on the performance of individual elements or variates.
This is another form of A/B test where you are simply redirecting the variation traffic to your newly developed page(s). One important thing to consider when setting up a redirect A/B test is to use a rel=canonical tag. This will tell Google and other search engines that the content is similar to the original URL and that your SEO ranking is not penalised when running the test.
What is Server-Side testing?
Most commonly used A/B testing methods are client-side – i.e. the changes are applied on the browser using JavaScript. This does not require any code push from the server and marketers and A/B testing developers can rely on the testing tool to set up the A/B test.
Client-side testing allows users to test almost everything. There can be cases where a hypothesis might be testing additional features – such as the pricing structure, new site search algorithm, anything that will require backend changes to the site and you cannot test them with the client-side testing. This is where server-side testing becomes handy. The backend developers can create the variation, and with the help of the testing tool, they can launch the test by splitting the traffic.
For example, let’s say Neil identified that the site search on the Best Retailer website is not returning the right set of results. The developed built a new search algorithm. But Neil wants to make sure that the new search algorithm is actually performing better in comparison to the existing search.
In this case, he requests the AB Testing development team to create a server-side test and using the server-side testing tool, the development team can launch the new version of the search alongside the existing version by splitting the traffic.
There are 5 core components of an A/B test:
Test Plan or Test Brief
Design of the A/B test variations
Development of the variations
Tool Setup
QA
1. Test plan or Test Brief
Also sometimes known as ‘Blueprint’, a test plan or test brief should provide all the necessary information about the A/B test. It should include, but not limited to, the hypothesis, audiences, targeting conditions, URL where the test will be launched, variation information, metrics that need to be tracked, QA scenarios – i.e. anything that would define any aspect of this A/B test will need to be documented on the Test Plan. This should be treated as the reference points for everyone involved in this CRO programme to learn more about an individual test.
You can download a test plan template from here[1].
2. Design of the A/B test variations
Depending on the changes being made with the test, the design of the variations needs to be specified. Ideally, it is great if the A/B testing developers get access to the raw design files (e.g. Photoshop, Figma, Zeplin etc). That way, when the A/B Test developers are building the variations, they can rely on the exact specification from the raw file (e.g. colour code, pixel amount etc.).
If the A/B test needs to be developed across all devices (e.g. Desktop, Tablet, Mobile), then the design should specify how the variation will be displayed across all of those devices. Additionally, if there are specific scenarios that would impact how the variation will be displayed (e.g. clicking on an accordion/tab), the design should also provide information such as active state, highlighted state.
3. Development of the A/B test variations
Once the test plan and design variations are ready, A/B test developers or Solutions Engineers start the development of the variations. Depending on the changes that are being made with the variations, the test developers would write the code using JavaScript / CSS / HTML. They can use the Browser Console to run the code and check if the desired changes are happening or not. Once they are happy, they can move into the tool setup.
4. Tool Setup
Depending on the testing tool, the setup of an A/B testing tool may vary. Additionally, the type of test can also dictate the setup of the test. For example, if it is an MVT, the developer needs to define the area where the changes are being made. If it is a redirect test, then the right parameters need to be used.
The A/B testing developer also needs to think about the targeting conditions based on the test plan. Who should be targeted, where is the test going to be live, if the site is developed using SPA, what needs to be done to ensure that the experiment is running at the right location.
Metrics setup is another important part of this stage of A/B testing development. The developers need to go through the list of metrics and implement them. Note that in some testing tools, you can save the metrics and re-use the saved metrics. Whereas in some other tools, every time you create a new test, you need to create the metrics.
When implementing the metrics, A/B testing developers would need to make sure that they are tracked in variations, as well as in Control. Once this is done, the test developer will need to prepare this for QA of the variations.
5. A/B Test QA
Once the development is complete, the test must go for QA. Test QA Engineers at this stage would pick up the test plan, check the variations against the design and the test plan to make sure that the variation is rendering as it should. They will report all bugs, which will then need to go back to the developer to fix. Furthermore, once the developer fixes the bugs, they will need to go through a re-QA process.
This is a significant part of any A/B test. Without QAing the test, the test should never go live to the end-user and you can read why A/B Testing QA is really important. The last thing you want is your test becomes invalidated by a bug in the variation.
QA Engineers also need to make sure that the metrics are getting the right data in control as well as in variation. Depending on the testing tools, the way of checking metrics can vary. However, in most cases, the metrics that are being passed to the testing tool server can be found on the network tab of the browsers.
Conclusion
A/B testing comes in different forms. However, all tests have some fundamental components – and they will define the quality of the tests that you are building for your CRO / Experimentation program.
A simple (yet powerful) guide to AB Testing Development
Over the past decade, companies had no choice but to take the online user experience more seriously than ever before, to increase their online sales, customer loyalty and achieve their business goals.
As the online businesses race towards becoming the best amongst their competitors, CRO and AB Testing have started to play a significant role to improve digital channel performances. From large companies such as Google, Facebook, Amazon, Netflix, Microsoft to Start-ups, started experimenting with different concepts to gain additional customers or users to their website.
In this article, we are providing a simple guide to develop A/B tests. However, before doing that, it is important to understand the basics, which will help to provide a solid foundation to understand the success of CRO.
What is CRO (Conversion Rate Optimization)?
There are many definitions out there of Conversion Rate Optimization. For example, Econsultancy defines CRO as a “process of optimizing site to increase the likelihood that visitors will complete that specific action”[1] . HubSpot provides an action-focused definition of CRO by adding enabling people to take action when they visit a website. “By designing and modifying certain elements of a webpage, a business can increase the chances that site visitors will “convert” into a lead or customer before they leave”[2].
In simple terms, the definition of CRO is
“Making changes to improve metrics.”
The changes can vary from small – like a simple headline change, to large – where you could be adding a new feature to your product. The changes can be on your website, emails, mobile Apps, Search keywords, banners, or even physical entities. The ultimate thing is by making these changes, you are improving your business metrics, performance indicators or KPIs. Moreover, this process of continuous improvement of KPIs by making changes is CRO.
[1] What is conversion rate optimisation (CRO) and why do you need it? https://econsultancy.com/what-is-conversion-rate-optimisation-cro-and-why-do-you-need-it/
[2] The Beginner’s Guide to Conversion Rate Optimization (CRO) https://blog.hubspot.com/marketing/conversion-rate-optimization-guide
What is A/B Testing?
As part of the CRO program, you are coming up with ideas to make changes. These change ideas are commonly known as Hypothesis.
As with any research, a hypothesis is always validated by different forms of testing. The most common form of testing a hypothesis within the CRO program is A/B Testing (or ABn Testing) where version A signifies the control or original – what is currently live, and version B is the new variation created based on the hypothesis.
A/B Testing simply helps you to check if the changes you are making are truly improving the target KPIs. It takes the metric performance of the current version and compares it with the new version where the changes have been applied.
For example, Neil – an Optimisation Consultant has completed some research on the e-commerce site he is involved with. Based on the analytics data, he identified that on the product details pages, the price of the product is presented just below the title of the product, which some users were ignoring. His idea is to change the position of this price and make it closer to the main Call to action (CTA) button so that users can see the price before adding the item to the basket and more people will complete their purchase through this website.
This is a simple example of an AB test to find out if Neil’s hypothesis is true – by moving the price closer to the CTA, more people will complete their purchase.
Figure 1: Example of a simple AB Test where Version A or Control where the price is just below the title of the product. Version B, where the price is just above the Buy now CTA button.
A hypothesis can create more than one variation. In such cases, your AB test becomes ABn test where A still signifies the control and after that, you have variation B, variation C and so forth to signify the other variation.
In case if you are interested in changing multiple elements and test a combination of different elements, you can setup a Multivariate Test (MVT).
Multivariate Test or MVT is a form of a test where you want to find the impact of the combinations of more than one change across multiple places within a page or site section. Let us break this definition down with an example.
Let’s go back to Neil’s research, and this time, he noticed that when visitors landed on the homepage of the best retailer website, they completely ignored the Sale link and also did not notice the key USP (see below).
Figure 2: Example of a Multi-variate test where two different elements are going to be tested with multiple variations
He wants to test out two different versions of the Sale message against the control version:
Control Sale message: Up to 50% off! Shop Sale
Variation 1 Sale message: Sale now on! Get Up to 50% off
Variation 2 Sale message: Up to 50% off! Hurry while stock lasts!
At the same time, two versions of the main heading to promote free delivery and return messaging against the control:
Control Heading: Welcome to the Best Retailer site – number 1 in sustainable products
Variation 1 Heading: Number 1 Sustainable Shopping site with FREE Delivery and Returns
Variation 2 Heading: Free Delivery and Returns! Best Retailer for Sustainable products
Neil is interested in finding out not only which individual sale message or heading works best to increase the browsability and purchase, but also to see which combination of these messages work best. This is where he sets this test up as a multi-variate test or MVT to test out all of the combinations against the control.
All MVT or multi-variate tests are AB tests – with a lot of variations. For example, the total number of combinations in this example will be 9 (including the control). Each of these combinations can be considered as a variation of an AB test.
Testing tools that provide the option of setting up MVT usually gives the user some additional options – for example, excluding specific combinations and reporting on the performance of individual elements or variates.
This is another form of A/B test where you are simply redirecting the variation traffic to your newly developed page(s). One important thing to consider when setting up a redirect A/B test is to use a rel=canonical tag. This will tell Google and other search engines that the content is similar to the original URL and that your SEO ranking is not penalised when running the test.
What is Server-Side testing?
Most commonly used A/B testing methods are client-side – i.e. the changes are applied on the browser using JavaScript. This does not require any code push from the server and marketers and A/B testing developers can rely on the testing tool to set up the A/B test.
Client-side testing allows users to test almost everything. There can be cases where a hypothesis might be testing additional features – such as the pricing structure, new site search algorithm, anything that will require backend changes to the site and you cannot test them with the client-side testing. This is where server-side testing becomes handy. The backend developers can create the variation, and with the help of the testing tool, they can launch the test by splitting the traffic.
For example, let’s say Neil identified that the site search on the Best Retailer website is not returning the right set of results. The developed built a new search algorithm. But Neil wants to make sure that the new search algorithm is actually performing better in comparison to the existing search.
In this case, he requests the AB Testing development team to create a server-side test and using the server-side testing tool, the development team can launch the new version of the search alongside the existing version by splitting the traffic.
There are 5 core components of an A/B test:
Test Plan or Test Brief
Design of the A/B test variations
Development of the variations
Tool Setup
QA
1. Test plan or Test Brief
Also sometimes known as ‘Blueprint’, a test plan or test brief should provide all the necessary information about the A/B test. It should include, but not limited to, the hypothesis, audiences, targeting conditions, URL where the test will be launched, variation information, metrics that need to be tracked, QA scenarios – i.e. anything that would define any aspect of this A/B test will need to be documented on the Test Plan. This should be treated as the reference points for everyone involved in this CRO programme to learn more about an individual test.
You can download a test plan template from here[1].
2. Design of the A/B test variations
Depending on the changes being made with the test, the design of the variations needs to be specified. Ideally, it is great if the A/B testing developers get access to the raw design files (e.g. Photoshop, Figma, Zeplin etc). That way, when the A/B Test developers are building the variations, they can rely on the exact specification from the raw file (e.g. colour code, pixel amount etc.).
If the A/B test needs to be developed across all devices (e.g. Desktop, Tablet, Mobile), then the design should specify how the variation will be displayed across all of those devices. Additionally, if there are specific scenarios that would impact how the variation will be displayed (e.g. clicking on an accordion/tab), the design should also provide information such as active state, highlighted state.
3. Development of the A/B test variations
Once the test plan and design variations are ready, A/B test developers or Solutions Engineers start the development of the variations. Depending on the changes that are being made with the variations, the test developers would write the code using JavaScript / CSS / HTML. They can use the Browser Console to run the code and check if the desired changes are happening or not. Once they are happy, they can move into the tool setup.
4. Tool Setup
Depending on the testing tool, the setup of an A/B testing tool may vary. Additionally, the type of test can also dictate the setup of the test. For example, if it is an MVT, the developer needs to define the area where the changes are being made. If it is a redirect test, then the right parameters need to be used.
The A/B testing developer also needs to think about the targeting conditions based on the test plan. Who should be targeted, where is the test going to be live, if the site is developed using SPA, what needs to be done to ensure that the experiment is running at the right location.
Metrics setup is another important part of this stage of A/B testing development. The developers need to go through the list of metrics and implement them. Note that in some testing tools, you can save the metrics and re-use the saved metrics. Whereas in some other tools, every time you create a new test, you need to create the metrics.
When implementing the metrics, A/B testing developers would need to make sure that they are tracked in variations, as well as in Control. Once this is done, the test developer will need to prepare this for QA of the variations.
5. A/B Test QA
Once the development is complete, the test must go for QA. Test QA Engineers at this stage would pick up the test plan, check the variations against the design and the test plan to make sure that the variation is rendering as it should. They will report all bugs, which will then need to go back to the developer to fix. Furthermore, once the developer fixes the bugs, they will need to go through a re-QA process.
This is a significant part of any A/B test. Without QAing the test, the test should never go live to the end-user and you can read why A/B Testing QA is really important. The last thing you want is your test becomes invalidated by a bug in the variation.
QA Engineers also need to make sure that the metrics are getting the right data in control as well as in variation. Depending on the testing tools, the way of checking metrics can vary. However, in most cases, the metrics that are being passed to the testing tool server can be found on the network tab of the browsers.
Conclusion
A/B testing comes in different forms. However, all tests have some fundamental components – and they will define the quality of the tests that you are building for your CRO / Experimentation program.
