Build Credibility by Measuring the Impact of Learning

Before the training department became L&D, we focused on internal measures. We had only classroom training augmented with films and workbooks. We measured attendance, asked how participants liked the training, gave them a test, and declared success.

Our world has been turned upside down. Business leaders want to know learning is useful to the organization. CEOs want to know the impact, and CFOs want to see a return on learning investment.

The first challenge is to isolate the effects of enterprise learning. When it does not happen in a controlled environment, identifying causation can be challenging. When you are trying to improve sales skills with learning technology, you will likely have marketing campaigns, new products, and new channels affecting sales. The owners of those initiatives will have protective owners who have their own claim to adding value. ^[1]

Isolation Techniques

Unless our measurement methods isolate learning as the differentiating factor, we cannot make a solid case for added value. Merely showing a link is not enough. We need to present a credible case for causation. 

Control Group. In scientific inquiry, a control group is the most credible method for measuring behavior changes and their impact. However, it has inherent difficulties that make it rarely suitable for a business environment. Among those are:

• the problem of finding two or more groups with the same environmental influences,
• inability to define the factors that affect outcomes,
• getting buy-in for control groups when their leaders want to get the benefits of improvement programs, and
• the difficulty of isolating groups, which creates the risk of cross-contamination, where the control group adopts the behaviors of the experimental group.

If you don’t have support for a control group, you can try using a pilot program. It can be easier to get buy-in from business leaders than if you appear to be depriving one of more groups for the sake of measurement.

Trend Line Analysis is a technique borrowed from statistical process control in quality management and familiar to Six Sigma practitioners. If you have a stable process where variation is within control limits, you can compare results after the learning intervention to assess the results. New variables introduced during the measurement period will affect the results.

Forecasting. Advanced analytical tools can help manage multiple variables. For example, if your data analysts can state with 90% confidence that the learning intervention accounted for 40% of the result, you have a sound numerical basis for calculating ROI.

Expert Estimation can carry a lot of credibility. It is the method we recommend in our e-book   By forming a partnership with the line of business leader who owns the process, you can work with that manager to develop an estimate of the impact. Using this method, you rely on the credibility of the person who manages the process. The result of your analysis will be an ROI estimate.

Focus Groups can work for you if you have the resources to conduct them. Using statistical analysis, you can develop a confidence level in focus group. For example, if the focus group showed a 10% improvement in market growth with 50% confidence, the adjusted percentage of the increase because of the learning intervention is 5%.

Questionnaires are another resource-intensive method for gathering impact results. They can be useful if focus groups are not workable.

Form a Business Partnership

Regardless of the method you use, forming a partnership with the line of business leaders who own the results will give you substantial credibility in your measures. Assume a supporting role in performance improvement and let them take the lead. It won’t be long before they view your L&D organization as one of the primary factors driving business performance.

References: 

1. Elkeles, Tamar, Ph.D., Patricia Pulliam Phillips, Ph.D., and Jack J. Phillips, Ph.D. Measuring the Success of Learning Through Technology. ASTD Press. 2014.

PhenomeCloud is a comprehensive technology solutions provider committed to empowering businesses to overcome challenges, enhance their workforce capabilities, and achieve superior outcomes.