2. The Phillips Model of learning evaluation
A major shortcoming of the Kirkpatrick Model is that it stops just short of providing a true training ROI cost-benefit analysis. You can show that the training produced measurable results, but how does that stack up against the costs of running the program? To fix that, we can look to the Phillips Model, which builds on Kirkpatrick’s framework with a fifth level: ROI.
The five levels of the Phillips Model are based on Kirkpatrick’s Model, with a few tweaks meant to create more data, and context around that data, to ultimately help determine ROI.
- Level 1: Reaction — Like the Kirkpatrick Model, you gauge your employees’ reactions to their training experience.
- Level 2: Learning — Like the Kirkpatrick Model, you assess whether employees’ retained the knowledge they received during training.
- Level 3: Application and Implementation — Instead of just identifying a change in behavior, the Philips Model accounts for factors outside of training that might help or hinder progress.
For example, maybe employees started using the new sales techniques, but the organization pivoted from phone sales to in-person sales. Level 3 in the Kirkpatrick Model would just say the learning was not implemented, but the Phillips Model gives you space to explain why.
- Level 4: Impact — Impact is a more holistic measure than the Results level of the Kirkpatrick Model because it takes into account both the positive and the negative impacts of training on the organization. Here you’ll assemble all the financial benefits caused by training, a well as all the costs associated with it.
- Level 5: ROI — Finally, how does impact translate financially? In this step, you’ll do a cost-benefit analysis to determine the ultimate value of the training program. You’ll compile all the financial impacts and all the potential costs of the program to discover its true financial benefit.
To calculate training ROI, you would collect the data for levels 1-4, creating your chain of evidence, just like in the Kirkpatrick Model. The only major difference is that, instead of calculating one number for Level 4, you would attempt to capture impact figures for a variety of metrics and would convert them into a dollar value.
So, for our sales training example from the first section, let’s say we were able to connect our training program with a 10% increase in average order value across the company, translating to a $200,000 increase in sales over the next year.
That’s great. But impacts can also be negative. How much did it cost to run the training? Consider these factors:
- Resource Development Costs — How much it costs to create and implement the course, including the price of an LMS. In this case, let’s say we spent $10,000 on developing our course.
- Time Spent — The time employees spend completing training. In this case, we trained 100 employees, who each make $25 an hour, and that training took three hours. Total: $7,500
- Implementation Costs — Other costs to run the course—for example, technical support, equipment, or space rental. However, our course was run online, and we didn’t need to hire additional tech support, so our implementation costs were $0.
So, in this scenario, our total cost was $17,500, and our total benefit was $200,000.
The equation for calculating ROI is quite simple:
So our ROI is 1040%. A pretty impressive number that shows we got a healthy return on the money we spent.
Remember that when it comes to calculating ROI, timing is everything. Don’t wait until the program is completed to start calculations or you may find that your ROI is smaller than expected or nonexistent. Instead, do the math at intervals throughout the training process so you can adjust training or implementation to make sure you’re getting maximum value.
3. Brinkerhoff's Success Case Method
Data backed doesn’t always mean hard numbers. While the two methods above focus on gathering quantitative data that proves learning effectiveness, Brinkerhoff’s Method aims to gather qualitative evidence. Instead of calculating a dollar value for training, you’ll create compelling examples and case studies to help sway decision-makers toward favoring your programs.
Because Brinkerhoff’s Method focuses less on proof of learning and more on the actual impact of education, you don’t have to spend so much time compiling statistics and tying actions to outcomes. There is no guesswork for isolating variables because we’re only looking at the outcome.
The Success Case Method is a simple way to combine program analysis with case studies and storytelling. It works like this:
- Identify training goals and expectations. What results do you hope to see as a result of the training experience? What does success look like?
- Identify outliers. Survey or evaluate training participants to identify those who have done the best job of meeting expectations and those who have done the worst.
- Research. Conduct in-depth interviews with individuals from both groups to document the impact of this specific training on their work and to identify what factors made the course successful or unsuccessful for them.
- Document your results. Use your findings to create two kinds of deliverables: case studies that profile the most compelling success stories, and a report that showcases the strengths and failures of the training program and where you can potentially improve for next time.
So, to evaluate our sales training, we would start by picking our goal metric. Let’s stick with our example of average order size per call. While the average increase in cart size was 10%, five salespeople went above and beyond and boosted their numbers by upwards of 20%. There were also five salespeople who showed no improvement or even showed a drop in sales figures over that period.
So we sit down and hash it out with both sets of people. We ask them for their thoughts on the training, asked them what worked and what didn’t, and then asked them how that has been affecting their behavior.
From the top performers, we learn what they found most effective about the sales training and how it helped boost their performance. We come away with some impressive firsthand stories on how useful training was.
From the poor performer, we learn that they were confused by some of the sales techniques or that they are implementing those techniques incorrectly. From that, we take away some concrete ideas for improving the program next time around. We pair that knowledge with analytics data pulled directly from the training program to see exactly where users struggled during training.
Putting all of that information together, we can tell a persuasive story of how our training is instilling confidence and knowledge in our top salespeople, as well as provide some concrete ideas for how to make our next round of training even more effective. When presented with confidence, these stories can be just as persuasive as numbers in a spreadsheet.
Related: 3 Essential Tips to Create Engaging Training for Your Sales teams
Try our training ROI calculator
Still not sure how to best demonstrate the monetary benefits of quality training? We’ve developed our very own ROI calculator to help your L&D team forecast potential ROI. Simply plug in your numbers and we’ll do the math for you. We use this calculator internally, but for the first time, we’re making it available for anyone to use to calculate the impact of your training in various use cases, including:
- Sales enablement/training
- Retail training
- Digitalized in-person training
- Training for the manufacturing industry
- SaaS client training
Editor's note: If nothing happens when you click on the links or the banner, you might need to turn off your ad blocker to allow our bot to pop up. :)
Our ROI calculator is closely related to Level 5 of the Phillips Model of Evaluation, in that it will help you calculate a dollar-based ROI number. You can then use any of these three models to tie those numbers back to your learning initiatives.
Additionally, if you use Salesforce, 360 Learning has an integration that can help you measure ROI by drawing direct correlations between training and performance data, such as revenue or service tickets.
Make a plan to stay on top of tracking ROI
Which of these three methods is the most effective? That depends on your company, and the kind of data your decision-makers respond best to. Some may find personal narrative more compelling, whereas others require hard data to fuel their decisions. In most cases, a combination of both qualitative and quantitative data will be most persuasive.
When it comes to demonstrating ROI, it’s important to be proactive. By the time higher-ups are asking to see numbers, your program may already be on the chopping block. You should be measuring the effectiveness of your programs constantly and showing value whenever possible. That way, you can maintain company buy-in and keep your programs running.