With global investments in training and development forecasted to grow from $163 billion to $487.3 billion USD by 2030, measurement and evaluation (M&E) is crucial to ensuring the learning industry is doing things right (Business Wire, 2023).
However, demonstrating the return on investment (ROI) of a learning program or collecting data to inform a learning strategy can be challenging. That’s often why M&E is deprioritized – a lack of time, budget and resources to put in the work.
So, is there a way we can make M&E easier to prioritize? Enter artificial intelligence (AI): a promising new measurement and evaluation assistant.
Generative AI is AI that generates text, images or other data in response to prompts, and it can play many roles. It has made waves in the learning and development (L&D) industry by expediting content development … but content and timesaving are only one part of the picture.
In reality, generative AI can be used in many applications, such as building an M&E strategy by:
To illustrate this point, let’s look at a simplified, fictional case study of Micah, an L&D manager who is designing an M&E strategy to support commercial product launch training.
Following Micah, you’ll see how AI can lighten the load that comes with creating and executing your M&E strategies. You’ll discover that AI is not just a tool for increasing efficiency, but also for enhancing decision-making, amplifying creativity and more.
As you may have already experienced with AI, you’ll rarely get exactly what you need in only one prompt. You need to be detailed, strategic and persistent with your prompts to get a valuable result.
It is also crucial to understand how to best protect yourself and your data. You should always refer to your company’s AI and data privacy policy before using a new tool.
When we think about M&E and AI, we often focus on the results, yet they are only one part of the bigger picture. When you begin to consider how AI can support you in all stages — conception, strategy, design and execution – and learn how to use and prompt AI effectively, the possibilities grow exponentially.
So, let’s continue with Micah’s story!
Micah’s first task is to define the learning and business objectives for the new training program.
She turns to AI to brainstorm common learning objectives for a sales training program. Her first few outputs are vague and non-specific.
Micah tries again, this time asking AI to put itself in her shoes and consider what the focus should be for sales representatives in an oncology therapeutic area. The new outputs are better, and after some editing, she ends up with focused objectives that she shares with senior stakeholders overseeing the program.
Next, she uses AI to review and suggest improvements to the goals to ensure they are measurable. She refines and iterates her prompts to develop a set of robust learning and business objectives with defined endpoints and metrics to track.
From there, Micah begins to map out the learning program. She asks AI to recommend an M&E strategy based on her training program goals and uses follow-up prompts to refine the output based on data collection methods she has access to, her discussions with the sales director and what has been done previously.
The recommended strategy involves a mix of key performance indicators (KPIs), assessments, stakeholder interviews and informal feedback gathering. Feeling overwhelmed, Micah turns to AI for help to get started with designing the M&E assessments.
After a year, the training program is well underway. Micah has launched a post-training assessment survey, has collected KPI data and is now ready to analyze the results.
She inputs the data into AI to conduct an analysis of numerical data and determine statistical significance, evaluate whether her goals were met and generate recommendations for future enhancements and initiatives. The program returns with quantitative results for numerical data and a summary of qualitative results by pulling key takeaways, feedback and quotes from the learners, saving Micah countless hours analyzing the results herself.
There are many ways that AI can support the full measurement and evaluation process to ensure a focused approach, generate the right data and analytics and demonstrate results. Through critical thinking and iterative design, you can use AI to give you a boost in productivity.
However, AI isn’t perfect, and it isn’t always easy to use. You should expect to invest some extra time up front to learn what types of prompts work best for your needs.
Most importantly, you should always remember the limitations of AI and refresh yourself with your company’s AI and data privacy policies before exploring.
Here are two top tips:
Remember, learning how to best leverage AI is a journey, but if learning professionals can invest the time up front, they have a real opportunity to advance industry M&E capabilities.
With some practice, iteration and curiosity, AI can introduce efficiencies in your day-to-day, drive decision-making, and expand your creativity and the scope of your capabilities.
Let 2024 be the year you explore AI! We know it’ll be worth the investment.
Julia Hatherell is manager, healthcare marketing & learning strategy, for Metrix Group. Email her at jhatherell@metrixgroup.com or connect through www.linkedin.com/in/julia-hatherell/.
Natasha Ilincic is manager, medical writing & learning design, for Metrix Group. Email her at natasha.ilincic@metrixgroup.com or connect through www.linkedin.com/in/natasha-ilincic/.
