It’s almost go-live, and your team is finishing up. Unit testing is done, integration testing was successful, the system is working, and now it’s time to let the end users try it on for size… How do you organize so many stakeholders to walk through what you’ve built? How do you engage them, keep them happy, and ultimately get their approval?
A UAT Party is in order!
User Acceptance Testing (UAT) is usually the last step before go-live, and is critical to both the project’s success and managing the user side of system change. It gives users an opportunity to interact with the new functionality before they have to make the switch, and gives the project team valuable feedback about how the system will really be used.
While Unit Testing answers the question: “does it work?” and Functional Testing answers the question: “does it work as expected?”, UAT answers the question: “can I use it?” This represents a very different set of criteria beyond technical and functional requirements, and uncovers the objective opinions of the users. With that in mind, structuring the UAT process in a way that engages your participants is critical to getting the feedback you need, and cultivating champions for the upcoming change.
Everybody who’s anybody will be there
Involving participants from every stakeholder group will ensure a broad perspective across your organization. Ideally, invite 2-3 participants who can represent the business needs of each stakeholder group. These individuals should have specific business knowledge that applies to the function being tested, and the ability to articulate the system’s use across all levels of their stakeholder groups. Keep in mind the invitee list can be long, but manageable, with proper planning.
Hosts with the mostest
Your participants will need direction, guidance, and expertise to help them walk through your test scenarios. The best hosting teams include Business Analysts who know the functionality inside and out, a technical counterpart who can speak to the architecture of the system, and Change leadership or training facilitators. Although the team is meant to help the testers, they can also obtain valuable insight from the testing party. An Architect has an opportunity to see her technical design in action, an Analyst can watch his use cases play out, and the Facilitator can refine the instructional design based on questions asked during testing.
Location, Location, Location
Select a comfortable space with enough room to allow your testers to move around. If possible, a location away from the normal office space is great - it gives participants a physical distance from urgent tasks and ringing phones that can be distracting during testing. Bring laptops and extra power strips; users will need to be hands-on during testing, and having computers booted-up and ready to go with the test environment running will help everyone jump right in.
Entertainment - the feature presentation
Well… nobody ever said that testing was entertaining… So let’s talk about keeping your testers engaged. Ideally you will want them dedicated for a chunk of time; a few hours, say. You will need to feed and caffeinate them. Everybody loves free food, and you are asking your testers to learn new routines and explore new processes, which requires concentration and can be draining. Snacks give participants an excuse to move around, take a little break, recharge, and keep up with the pace of your party.
Speaking of pace, choreographing the party is key. Here are 5 tips to consider when planning:
Set their expectations - Tell your testers about the agenda and outcomes prior to the event, so they know what to expect when they arrive. Kick-off the party with an overview of the functionality to be tested, and introduce the resources you will be using during testing (documentation, use cases, test scripts, feedback forms, etc).
Give them tools and space - Everybody tests differently, and giving participants a range of support materials will help them feel comfortable with the task at hand. Some users enjoy testing in small groups to divide-and-conquer or bounce ideas off one another. Others need silence and checklists or training documents. For UAT, I have found that providing use cases explaining real-world business scenarios works better than test scripts like those used for functional testing. With a use case, testers can walk through the steps they would normally do at their desks, making it feel more like what they will do after go-live.
...But keep it structured - System-wide UAT can be overwhelming. Consider breaking it into smaller parties, each with a theme: Revenue Reports, Solicitation and Communications, or Significant Donor Life Events (marrying, deceasing, and adding family members), for example. Provide timed segments with breaks between. Set goals for each segment, and re-engage the group by asking for initial feedback at the end of each segment.
Show them, and show them again - Although UAT is not user training, it is an opportunity to give users their first experience with the new tool. Ask your hosts to demonstrate the functionality to the entire party, multiple times if needed. One-on-one time with each participant is important, too. Hosts should walk around during the party, checking-in with testers. The questions asked one-on-one will be useful business-specific tidbits during training.
Did I mention food? Just please don’t run out of coffee...
A graceful exit
Collecting defects during UAT will help support the process to package and deliver the solution, and uncovering any late-breaking additional business requirements will establish a roadmap for future enhancements. It is fundamental that your testers understand what recommendations you categorize as a critical defect (needing attention prior to go-live), a non-critical defect (needing attention, but not prior to go-live), and an enhancement (new or expanded functionality); the line can sometimes blur. Set an expectation with your party-goers about the timeline for each recommendation type so that they feel comfortable providing it. If they believe an enhancement will never be a priority, they will insist it is a critical defect and will block deployment until it is complete.
Ultimately, you need to get approval from your testers, so understanding how they prefer to sign-off is key. Choose to either collect approvals from each individual, ask for a blind vote, or funnel feedback into the project Sponsor for final approval. As long as the project team and the testers understand the strategy at the start, collecting feedback should be straightforward.
The after-party
Believe it or not, large-scale UAT can be successful and even a little fun. You are asking your users to partner with you at the very end of a project; to give you a valuable perspective that will help the launch be exciting for the greater population. Give your testers solid expectations, the tools they need to help you, and a structured, time-boxed experience, and everyone will leave with new insight. Celebrate your success with a real party at go-live!!
Enlist the help of the party planners at BrightVine Consulting the next time you develop a strategy for Testing and Change Management. We’ll bring the cookies! Contact us for more information.