Evaluating L&D in a Knowledge Economy

Evaluating L&D in a Knowledge Economy 

Contents

Evaluating L&D in a Knowledge Economy

1.1 Understand the meaning and purposes of evaluation within a learning and development context.

Slide 3 – Define evaluation

Slide 4

1.2 Explain how evaluation fits within learning and development processes and systems

1.3 Discuss the importance of learning transfer into the workplace

Importance of a knowledge economy

Bibliography

1.1 Understand the meaning and purposes of evaluation within a learning and development context.

Slide 3 – Define evaluation.

Assessment is the process of gathering data. In L&D, assessment is the way an instructors, for example, gathers data about their teaching and their students’ learning.  The data provides a picture of a range of activities using different forms of assessment such as: pre-tests, observations, and examinations. Once this data is gathered, it is possible to evaluate the student’s performance.

 Evaluation, therefore, draws on one’s judgment to determine the overall value of an outcome based on the assessment data.

It is in the decision-making process where we devise ways to improve any recognized weaknesses, gaps, or deficiencies. 

It is a systematic process of assessment, evaluation, and decision-making. The results (data) of the assessment (examinations, observations, essays, self-reflections) are evaluated based on judgment of those data.  The decision making step, is based on the evaluation.

        (Formative and Summative Assessment)

To evaluate effectively we need to know why the training is taking place, what it is seeking to achieve and any wider or organizational factors that might impact on it, as well as what impact it has on the business.

Slide 4

There are two types of assessment.

Formative – Formative assessment provides feedback and information during the training process, i.e. while learning is taking place, and while learning is occurring. Formative assessment measures student progress.  A primary focus of formative assessment is to identify areas that may need improvement.

Summative – Summative assessment happens after the learning has taken place.  This evaluation assesses the value and effectiveness of the intervention; identifying its continuance or modification.

The aim of evaluating a learning intervention is to reflect of the journey of not only the individual but also the business.

By evaluating you can identify:

Process – whether the intervention has been implemented as intended and whether it has achieved what it intended.  This is more what and how of the teaching that takes place.  Was the programme followed? Should the programme be adjusted with time and knowledge? Did the programme achieve its goal?

Outcome – This evaluation focuses on the change in knowledge, attitude and behaviour as a result of the intervention. It measures the abilities of the participants after the training and did they apply these new skills back in the work place. Are staff more confident after the training? Do they apply what they have learnt in the work place?

Impact – An impact evaluation looks at the effectiveness in achieving its ultimate business goal; by looking at how well the program achieved its learning objectives and whether the training affected the overall business strategic goals. Return of investment -  can a financial or economic benefit be attributed to the learning invention in relation to the cost of the training programme.

1.2 Explain how evaluation fits within learning and development processes and systems

Training Needs Analysis – Getting an understanding of the strategic aims and goals for the organisation which allows L&D to start assessing what learning and training needs are required in the business over the next 12 months or so. 

• Talking to senior stakeholders and discussing organisational plans such as growth, diversify, merger, acquisition, expansion into new markets, development of new products.  Using this information L&D can assess the needs on an organisational level and assess any skills gap,

• Meet with heads of departments to understand what the organisational plans will mean practically and what their training needs are, being aware that each department may require different needs

Design Planning – L&D are looking for a solution to support the business in its training needs.  Looking at what type of training has been successful in the past, can it be developed further or is a more blended approach more effective.   L&D will also be looking at how they would evaluate the success of the training and meeting the objectives.

Deliver Training –

1.3 Discuss the importance of learning transfer into the workplace

“Learning transfer can be defined as the ability of a learner to successfully apply the behaviour, knowledge, and skills acquired in a learning event to the job, with a resulting improvement in job performance.”

           (Leaman)

Or it can be seen that “learning transfer can be defined as the sustained behavioural change that happens after learning, turning a cost into an investment.”

(Learning Transfer) 

Businesses send staff on training courses because there is a gap in their knowledge. People are employed to fulfill a role. Not only must they have the appropriate resources; they must also have the relevant knowledge, skills and attitude. Businesses invest in training for their staff to realize a positive return on their investment. Financially, the return on investment means less money out and more money in.  Training transfer is the realisation of that investment.

1.   Positive Transfer – the attendees return to the workplace and perform better than they would have without the training. Solving problems they couldn’t before, or they are more productive, more efficient, more engaged, more motivated.  Results are noticeable, measurable and desired.  The training has added value to the business as a result.

2.   Negative Transfer – In this instance the training has had a detrimental effect which results in training participants performing worse than they would have had they not gone to training at all.

3.   Zero Transfer – in this situation acquisition of new skills or new knowledge has had absolutely no effect in the workplace.  The productivity or performance of the staff who attended training is neither enhanced nor hindered. 

2.1 Compare and contrast key theories for evaluating learning and development activities.

The 3 theories chosen are:

• Kirkpatrick
• Behavioural
• CIRO

2.2 Explain the key steps within the evaluation process.

The key steps are:

1. Identify the purpose – clarify goals and objectives of the program
2. Key Stakeholders – Your stakeholders are supporters, implementers, recipients, and decision-makers related to your program. Getting people involved at the earliest stage will help you get different perspectives on the program and determine common expectations.

Taking the time to express what the program does and what you want to accomplish is essential to establishing the evaluation plan. It should answer questions like: What is the goal of our intervention? Which activities will we pursue to achieve our goal? What are our resources? How many people do we expect to include?

Evaluations can focus on process, means, resources, activities, and outputs. They can focus on outcomes or how well you achieved your goal. You may also choose to evaluate both process and outcomes

The two form forms are Qualitative and quantitative data.

Qualitative data offers descriptive information that may capture experience, behavior, opinion, value, feeling, knowledge, sensory response, or observable phenomena. Three commonly used methods used for gathering qualitative evaluation data are: interviews, focus groups, and participant observation.

Quantitative methods refer to information that may be measured by numbers or tallies. Methods for collecting quantitative data include counting systems, surveys, and questionnaires.

This is the step where you answer the bottom-line question: Are we getting better, getting worse, or staying the same? Data comparisons show trends, gaps, strengths, weaknesses. You can compare evaluation data with targets set for the program, against standards established by your stakeholders or funders, or make comparisons with other programs.

2.3 Identify sources of data and methods of analysis that can be used within the evaluation process.

3.1 Consider a number of key challenges that can act as barriers to conducting the evaluation of a learning and development activity.

1. Lack of funding or resources in the HR department – whilst senior manager see the investment of training their lack of insight and understanding of L&D as a whole prevents funding to see the whole project to completion.

2. Lack of evaluation expertise – L&D may not have the appropriate training or tools to evaluate the type of data that results from an intervention. It requires senior management to understand the importance of invest in this area.

3. Time – Whilst it is easy to say “Time Management” Business priorities change quickly and the HR/L&D departments have to move as quickly to keep up.  L&D concentrate more on other people within the business but forget ourselves.  L&D need time to teach themselves too.

3.2 Explore the characteristics of a knowledge economy and their implications for evaluating learning and development activities.

Characteristics of a knowledge economy:

1. Easy accessibility of data
2. Effective communication of knowledge within the organisation
3. Systems thinking
4. Human Capital
5. Intangible and tangible assets

Importance of a knowledge economy

It can help businesses be more efficient, dynamic and innovative.

1. Enables product innovation and customisation.
2. Greater role for human capital. Businesses need to attract and retain workers adapted to the new mode of economy.
3. To enhance knowledge distribution and benefit from new working practices collaborative networks can help.
4. New growth theories emphasise the potential for human capital and increasing knowledge to provide new sources of economic growth and high levels of productivity.
5. Potentially greater demand for skilled labour.

However, what it will reveal is a gap in knowledge, skilled labour versus unskilled labour. There is therefore a need to ensure staff at all levels are able to change and adapt.  Stakeholders must be trained in Change Management. Gaps in the knowledge must be identified with a training needs analysis. Evaluating what the business has and does not have.  It is therefore important to evaluate the effectiveness of the training at intervals… before, during, immediately after and 3/6/9 months down the line.  Up skilling and a growing demand for diversified competencies are key features of the knowledge economy.

L&D must also find a way to convert intangible assets such as soft skills into quantifiable tangible assets, such as motivation.

3.3 Discuss the key issues raised by intellectual capital accounting.

“… no doubt that successful companies tend to be those that continually innovate, relying on new technologies and the skills and knowledge of their employees rather than assets such as plants or machinery.”

(Starovic) 

The key issue raised by intellectual capital accounting is that it is seen as critical to most business with an increased focus on how it is measured and managed.

Whilst it is easy to put a value on a piece of machinery, it is not so easy to put a value on creativity.

Intellectual Capital is intangible so it has to be converted into something tangible… something that can be measured.

With Relational (Customer) Capital and how customers see the business and products they are given the opportunity to review new products and customer service on the company website or Trust Pilot, Facebook, Twitter and Instagram.  Creativity of a new product can be seen as a 5 star rating or how many “likes” an image gets on social media.  Google Analytics monitors traffic onto the website.  It can tell you which page of the website the customer is on, have they put anything in their basket, how long people stay on the website, where they have come from (web browser or directed by an advert) etc.  However, it needs to be programmed. It requires the knowledge and knowhow by Web Designers and Programmers.

The question that can be asked is “How much of the tech word does an L&D department know?”  Do they understand the technology and the language to be able to evaluate its needs and impact?  There is a tendency for the L&D department to develop other departments but do they include themselves in the learning and development process.  L&D may need some up skilling too!

Bibliography

• CIPD. (n.d.). Retrieved from Evaluating Learning Factsheet: https://www.cipd.co.uk/knowledge/fundamentals/people/development/evaluating-learning-factsheet
• Formative and Summative Assessment. (n.d.). Retrieved from Northern Illinois University, Faculty Development and Instructional Design Center: https://www.niu.edu/facdev/_pdf/guide/assessment/formative%20and_summative_assessment.pdf
• Leaman, C. (n.d.). Improving Learning Transfer. Retrieved from Training Magazine: https://trainingmag.com/improving-learning-transfer/
• Research & Reviews. (n.d.). Retrieved from Evaluation of Training and Development Programme: http://www.rroij.com/open-access/evaluation-of-training-and-developmentprogramme-.php?aid=47584
• Types of Evaluation. (n.d.). Retrieved January 2019, from The Peak Performance Centre: http://thepeakperformancecenter.com/business/learning/business-training/types-of-evaluations/