When "Boring" Means "Amazing": How Testing Makes Go-Live Day a Snooze
It was “go-live” day at Marketo – the day that our marketing automation platform would be turned on for several business units at a client company. We had been preparing for months. As the senior consultant on the project, I was on call to cover any issues that needed to be solved. Having witnessed explosive go-lives (second-hand, of course), during which everything had to be rolled back due to one unforeseen problem or another, I was prepared for the worst.
But at the end of the day, not a single problem had come up. More days passed, and I still didn’t hear about any issues from the client. “How could go-live be this easy?” I wondered. As it turned out, our rigorous user acceptance testing had saved the day!
Since then, I’ve observed that user acceptance testing is key to a peaceful – even boring – go-live day. I’ve also found that many enterprise clients don’t know how to design this kind of program.
Here's how to rigorously test, and provide some example test documentation to get you started.
Benefits of testing
Every implementation should include some level of testing. Here are some of the biggest benefits of testing that I’ve observed:
- Testing uncovers issues early, so they can be remedied prior to go-live
- Testing gives people an idea of how applications will actually function
- Testing serves as an acceptance exercise; client requirements can be demonstrated as met
- Testing is one of the best ways to train future users
What is testing?
You’ll want testers to discover how a program works, as well as any special features. With a marketing automation implementation, we want our testers to review how scoring works, how an email looks – anything that will be important at go-live. They might test an email marketing campaign, for example, or a lead nurture program. The testers use test scripts that guide them through testing.
A test script is a document that walks the tester through step-by-step testing of a particular functionality.
What does a test script look like?
Here are some example test scripts. Click the links to download the examples (they are Excel files)!
This script is designed to test your CRM sync with your marketing automation platform – for example, whether or not new leads pass in both directions properly: CRM Sync with Marketing Automation Test Script
This one is designed to make sure your lead lifecycle is functioning – i.e. lead status changes when expected, leads get to the reps when expected, email alerts fire off, etc.: Lead Lifecycle Function Test Script
This script is designed to test a marketing program: Marketing Program Test Script
Please note that these scripts will need to be adjusted for your use. As you can see, a good testing script is specifically tailored to each scenario.
How many scripts do I need?
In an extreme case, I had one client who used over 100 scripts and tested every single thing we built. My recommendation is to use a small number of scripts to test a good sample of your functionality. We used only three test scripts for a recent client, but each script was very important. These scripts were:
- Salesforce.com Sync – one script to test their Salesforce sync
- Lead Lifecycle – one script to test the lead lifecycle (most importantly, that leads got to the sales rep when expected)
- Marketing Program – one script to test the initial marketing program (if you have very different marketing programs, you may want to have a script for each type of marketing program)
So what steps need to be taken?
Here are the most important steps, as I see them:
- Create the test scripts. This is best done after the programs have been built. Until that point, you really don’t know what the script should be testing.
- Test your tests. Whoever wrote the test scripts should give them a test run – I guarantee that you’ll discover a need for revision. At this point, the campaigns should be activated, so that the tester can mimic the user experience.
- User testing. This is when the users actually run through the test scripts and perform the tests. You can do as many rounds of user testing as you’d like; I’ve seen as many as three rounds. A best practice here is to run through the test scripts with marketing operations once, and then have field marketing run through the scripts again. The field marketing round is considered the “user acceptance testing.” You will need to make adjustments to the scripts after each round, as marketing operations will want the scripts to be perfect for the field marketers. Have the users sign-off on the test scripts as well.
Sound stressful? It's not!
Testing may sound stressful, but it’s actually the opposite. If you invest the time to institute a formalized test program, you’ll make your implementation go smoothly. After all, when it comes to go-live day, boring is what you want!