How to measure eLearning ROI

Calculating eLearning ROI isn’t as easy as it sounds, but there’s a lot at stake if it’s ignored. American companies spent $92.3 billion on workplace training in 2021, and according to a Capterra study of HR leaders, 49% of companies said they would increase their L&D spending this year.

Measuring eLearning ROI can help learning and development teams provide stakeholders with evidence of its effectiveness to justify the investment. The Society for Human Resource Management states that measuring eLearning ROI also helps:

• Quantify the effectiveness of eLearning and identify areas for improvement
• Determine specific outcomes in terms of cost, time, and behavior changes
• Build trust and credibility for the L&D function
• Determine the overall L&D budget

However, a 2021 survey found that only 8% of L&D teams calculate ROI for their programs. If measuring eLearning ROI is so important, why aren’t more companies doing it?

Measure eLearning ROI

There is broad agreement in the L&D community that one of the biggest challenges in measuring eLearning ROI is the lack of standard metrics. The math to measure ROI is simple, but it requires identifying and quantifying the results in monetary terms, which can be far from easy. There may be disagreements within the L&D team about what benefits and costs should be included and how they should be calculated.

Historically, calculating ROI has focused on tangible assets such as time, salary, cash spend, revenue, and other factors that can be quantified and converted to dollar numbers. But how can L&D teams put a monetary value on intangibles like employee engagement, attitude change, or the impact of eLearning on an organization’s culture, mission, people, and brand?

How can L&D teams assign a monetary value to intangibles such as B. the impact of eLearning on culture?

It can also be challenging to isolate the impact of eLearning from all other factors affecting a company’s performance and results – current economic and market conditions, to name a few.

This uncertainty in quantifying costs and benefits can challenge ROI calculations, especially when L&D teams are unfamiliar with measuring eLearning ROI. Fortunately, L&D teams can find guidance in determining eLearning ROI in the following framework.

The Phillips ROI methodology

The most well-known model for measuring the effectiveness of training programs is Donald Kirkpatrick’s model, which has four levels: Response, Learning, Behavior, and Outcomes. However, Kirkpatrick’s model provides little guidance for assessing the extent to which a training program has met stakeholder expectations. It acknowledges that training reviews often “stall [level 4] because it can be difficult to develop appropriate quantifiable improvement actions.”

Jack Phillips, PhD – Director of the ROI Institute – thought this was a limitation that could be overcome. He built on Kirkpatrick’s 4-level model to develop his 5-level methodology. In truth, Phillips created a 6-layer model by adding layer 0 (inputs) and layer 5 (ROI). He also developed some strategies to isolate the effects of learning from the effects of other factors such as employee bonus programs, competitive environment, and seasonality that lead to temporary improvements in business performance.

Phillips developed strategies for isolating the learning effects, such as using control groups.

Such strategies include the use of control groups (comparing the performance of one group that was trained in a soft launch versus another group that was not trained), trend line analysis, forecasting, and estimation of program impact by participants, supervisors, and customers , and subject matter experts.

Level 0 of the Phillips model involves identifying direct and indirect cost elements that ultimately feed into the determination of e-learning ROI, such as: B. The organization’s investment in the learning program and the number of learners who have completed it.

Level 5 is divided into several stages for determining ROI: planning, data collection, analysis and reporting. These phases together comprise nine steps outlined below.

Stage 1: Evaluation planning

• Step 1. Develop project goals. Identifying the objectives of the ROI assessment is key to asking the right questions when collecting data
• Step 2. Develop evaluation plans. This refers to plans for data collection, data analysis and ROI calculation.

Stage 2: Data Collection

• Step 3. Collect data during and after project implementation. Use tools such as response polls, surveys, interviews, and focus groups according to the data collection plan.
• Step 4. Isolate the Training Effect. This step helps determine the scope of improvement resulting from the program and ensures credibility of the ROI calculation.

Stage 3: Data Analysis

• Step 5. Convert training results into monetary values. Hard data is easily converted and intangibles are identified below.
• Step 6. Record training costs. See the inputs identified at level 0.
• Step 7. Identify intangibles. If possible, convert to monetary value or report as unquantified intangible benefits.
• Step 8. Calculate ROI. More on this step later in the article.

Stage 4: Reporting

• Step 9. Develop a report for each audience and communicate the results.

Because this ROI calculation is preceded by the four layers – Response, Learning, Behavior, and Outcome Assessment – ​​L&D teams should be able to demonstrate that the business impact (positive or negative) was the result of the eLearning being assessed. If the ROI does not meet expectations or is negative, the previous levels can identify the cause.

Calculation of the eLearning ROI

The ROI is usually expressed as a benefit/cost ratio or as a percentage. The formulas are similar. Calculating the benefit-to-cost ratio (BCR) is a simple matter of dividing the total program benefit by the total program cost.

For example, suppose that the benefits from an internally developed e-learning program and completed by 260 employees totaled $1.2 million in the first year. And that the total cost of the program for the first year was $420,000.

The BCR would be 2.86 to 1. For every dollar spent, the organization’s training program yielded a whopping $2.86 in benefits.

However, the ROI covers the total costs by the total benefits. To determine eLearning ROI, divide the net benefit (total benefit of the program minus total cost of the program) by the total cost of the program. Then multiply the result by 100 to express it as a percentage.

Using the same numbers as in the previous example for total benefits and costs, the ROI calculation looks like this:

With an ROI of 186%, the net profit was $780,000 for a total return of $1.2 million. The return on every dollar spent was 186%.

When should you use BCR vs. ROI? BCR is commonly used to predict the expected benefits of a particular investment, while ROI is applied retrospectively once actual benefits have been realized and costs counted.

Calculating both the eLearning ROI and the BCR and comparing the two has the added benefit of giving you insight into the accuracy of the estimation process. One thing to be aware of is the possibility of bullish bias arising from the fact that people tend to exaggerate benefits and underestimate costs.

Benefits and costs of e-learning

Organizations are typically able to see the monetary benefits of eLearning in terms of cost savings compared to other training methods such as For example, you can calculate the monetary value of reducing lesson time by considering learner “seat time,” teacher preparation, platform, and follow-up time. The average reduction in teaching time and overall cost reduction compared to instructor-led training are reported to be approximately 60% and 50%, respectively.

The average reduction in lesson time and overall cost reduction compared to instructor-led training are reported to be approximately 60% and 50%, respectively.

The benefits of learning gains can also be quantified by measuring KPIs. However, some benefits may not be measurable. For example, surveying an organization’s workforce can reveal a quantifiable increase in morale and employee engagement, but attributing such improvements solely to eLearning is another matter entirely.

Phillips suggested that some benefits might simply need to be accounted for as intangible and not included in an ROI calculation, although they could serve as food for thought when evaluating eLearning initiatives. Some suggest using learner satisfaction ratings from post-training assessments as a proxy for such intangible benefits.

The costs associated with eLearning are usually easier to predict and quantify than the benefits. Technology-related costs include:

• Development or acquisition of digital content
• External or internal content hosting
• Distribution with external hosting
• LMS costs
• Purchase and maintenance of hardware and software

There are also eLearning costs associated with L&D employees, such as B. the costs of developing content, administration, consulting and providing support. These are easily quantified in terms of the number of hours of engagement by L&D staff. The same is true for the paid time employees spend on eLearning.

Some of the information required to allocate monetary values ​​may be obtained from systems maintained by human resources or the finance/accounting system. Organizations can also receive data useful for measuring eLearning ROI from their LMS, for example:

• Time that learners spend on individual courses
• Level of learner engagement
• results of the assessments
• Progress and course completion by individual learners
• Course Popularity
• Student feedback and survey results

There are a number of online tools that make it easy to calculate eLearning ROI. The ROI Institute offers free calculators for monetary benefits, program costs, ROI and BCR, and cost of sales. Another useful tool is a standard ROI calculator like this one.

final remark

Measuring eLearning ROI is the best way to prove its value. To make a compelling case for eLearning to stakeholders, the measures that matter most to the organization should be used. This can be the time it takes learners to complete an eLearning course or the time it takes L&D to convert a storyboard into a live module.

Identify which impacts of eLearning are most closely related to stakeholder expectations and place that at the center of efforts to measure ROI.