Meld u aan voor
onze nieuwsbrief

E-Learning Management with the E-Learning Scorecard

A method enabling an organization’s e-learning project leaders to implement project management disciplines. By Axel Lindhorst, Project Manager tts GmbH.

There are various possibilities for evaluating an e-learning project from a management perspective. The best known is without doubt return on investment (ROI), which sheds light on the cost-benefit aspects and establishes a simple balance sheet highlighting the savings that are made in the training environment (travel, training personnel, time) offset against the investment cost of the e-learning project.

A second approach is return on expectation (ROE), which measures the degree to which the project meets the participants’ expectations. Here, company leadership is included among the participants as well as the people doing the learning. In this case, you need to consider the benefits that the participants are hoping to get out of the project and how far their expectations align with the content and outcomes of the project.

The “E-Learning Scorecard” is a further method that Axel Lindhorst explores below in more detail…

What is an E-Learning Scorecard?

The E-Learning Scorecard is based on the Balanced Scorecard (BSC), which was developed at Harvard University in the 1990s by the two American economists Robert S. Kaplan and David P. Norton. BSC is a concept for measurement, documentation and control of a company or organization with regard to its vision and strategy. It caused quite a stir in the field of KPI-based performance management. The advantage lies in the holistic observation of cost and benefit factors beyond pure financials and the orientation around organizational goals and strategies. All of the objectives and KPIs that are identified for the BSC can be aligned to the overall company strategy. Around 40 percent of all German companies work – at least partly – with such scorecards.

The Four Perspectives of the E-Learning Scorecard

When you set up a scorecard, you need to choose four perspectives for measurement and analysis. These are not fixed – you can choose them for yourself. Typical scorecard perspectives for a project would be:

  • Finance: cost-benefit analysis
  • Stakeholder: project participants, including company leadership, the people doing the learning and other internal customers
  • Processes: internal business processes. What changed as a result of the new activity?
  • Innovation and knowledge: what is the impact on knowledge management, how do you handle innovative methods for imparting knowledge so that you can improve processes, satisfy customers and increase profits?

All four of these areas relate to one another. For example, when a company becomes more innovative, processes improve, customers become more engaged and profits increase.

The advantages of the E-Learning Scorecard

The advantages of a scorecard are manifold. On the one hand, it enables a flexible setting of priorities according to a department’s or project’s strategic prerequisites; on the other hand, it anchors the actions that your organization executes within the four managed perspectives. Moreover, the E-Learning Scorecard is also applicable to smaller projects, since it can be broken down to a project scorecard.

The greatest advantage is undoubtedly that it provides a medium for communicating the organization’s methods and approach. E-learning projects often find themselves under pressure to justify themselves to an organization’s leadership: “Where is the benefit? Why should we implement e-learning?” With the scorecard, the e-learning team can produce KPIs and thereby demonstrate the benefits of their projects.

What are the relevant KPIs and objectives in the four perspective areas?

1. Finance:

the objectives here concern commercial business, cost reduction resulting from the implementation of e-learning and efficiency gains. KPIs include costs per user and for the set-up of e-learning in comparison with other methods and infrastructure costs. What would be the necessary expenditure without e-learning, and how does that compare with the actual costs with e-learning?

2. Stakeholder:

the objectives of the learners and the managers involved in e-learning are clearly defined. Acceptance of e-learning should increase alongside satisfaction with the e-learning offer. Additionally, one would want to improve the homogeneity of the participants’ knowledge. To take the example of a software rollout, one of the outcomes would be for all participants to attain a similar level of knowledge of the system. The evaluation of usage statistics generated by a learning management system (LMS) can deliver the desired KPIs (for example, how many employees in total have taken advantage of the e-learning offer?) Other possibilities include the results of course evaluations carried out via questionnaires or interviews and test results from blended learning.

3. Processes:

Increasing productivity, the facilitation of process quality and the optimization of the technical infrastructure are the kind of objectives that belong under this heading. KPIs could include the level of demands on the technical hotline, service quality outcomes, media performance and availability of the e-learning offer. For example, if hotline enquiries increase significantly after a software rollout, you can address the problem with an e-learning initiative. If the number of enquiries then falls, you can take this as evidence that the initiative had the desired effect.

4. Innovation and knowledge:

The e-learning team has the objective of establishing e-learning practices in the organization and finding new approaches to knowledge transfer. This is the most important resource in terms of service delivery, for it allows you to demonstrate how a company can move from formal, “pushed” knowledge to informal knowledge, which employees can access from their own workplace.

Relevant KPIs include: the ratio of e-learning courses to the total offer; the number of times employees access e-learning modules (since this indicates the acceptance and innovation potential of the employees); the proportion of e-learning in overall knowledge transfer; and the number of knowledge transfer ideas put into practice, for example the implementation of social learning, blogs, wikis and so on.

In this scorecard, technical optimization is the given objective, measured by the termination rate (= the proportion of users who break off from e-learning within a specific time limit). The target KPI is a termination rate of less than five percent, but the current actual rate is 15 percent. Thus, the challenge is to find out why the rate is so high and what actions can be taken to improve it. One such action could be to increase the bandwidth available to the company network in order to improve the performance of e-learning courses.

Axel Lindhorst, Project Manager TTS GmbH

The objective in this scorecard on the stakeholder perspective relates to learning quality; the KPI is derived from the evaluation that the learners can give for example via a questionnaire (Happy Sheets). In this case, the average grading should be better than 2, but the actual average is 2.3. Once again, the challenge is to consider which actions would improve the score. One possibility would be to select the course offer more carefully.

Parameters

You should adhere to certain parameters when implementing an e-learning scorecard. Importantly, you should not define more than ten to 15 KPIs. It’s imperative that you focus on the objectives that you have identified as being most significant and aligned to the company strategy; otherwise you risk getting bogged down in unnecessary detail. In addition, you should be aware that you need to use a scorecard for a certain period before it reaches maturity –you need to go through the first reporting cycles to get the KPI scores. In general, this takes between one and two calendar quarters (three to six months). For one-off project scorecards, the reporting cycles could be shorter.

Chances and risks

Of course, a scorecard will entail risks, for example if you set too many complex objectives. Once again, it pays to focus on your core objectives. You should also take care not to set unrealistic expectations and responsibilities. The KPIs must be attainable, taking into account the cycle-times.

Deliberate manipulation is a further risk. If realistic and attainable KPIs are defined by the responsible parties and they are linked to the attainment of personal targets, there is a danger of manipulation for self-serving purposes. You can counter this risk through a broad distribution of objectives and perspectives.

Overall, the scorecard is an actionable approach to managing and controlling e-learning projects. It enables those in charge of such projects to present valid arguments to the organization’s leadership.