Measuring the effectiveness of the implementation process

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Measuring the effectiveness of the implementation process in bringing about the improvements in performance set out in the organization's strategy is clearly desirable. However, performance measurement is a particularly difficult challenge for social enterprises. There are conflicting and incoherent objectives in these organizations. Evidence of causal links between the adoption of strategic management and performance is often ambiguous. Tools available from the corporate world are often inappropriate for measuring the performance outcomes targeted by CSEs - social impact and sustainability. In spite of the above difficulties, the increasing demand for SEs to be accountable and to demonstrate their impacts is leading practitioners and academics to develop tools to measure economic and non-economic dimensions of performance.

The most important reasons to measure performance are 1) to improve organizational effectiveness in order to inform resource allocations in the future, 2) to acquire information to help any organisation and its stakeholders drive its objectives forward, and 3) to contribute to the development of best practice models to benefit the entire SE sector.

A particular difficulty in CSEs is that because it has an increasing role in the delivery of public services, it must measure not only its financial value but how it meets both its social and financial bottom lines. CSEs must perform according to "deeds and dollars," obtaining enough financial resources while fulfilling unmet social needs with implementation. These characteristics result to a dual role of CSEs: passionate implementation of targeted activities and sound business management for financial stability of the organisation.

Performance metrics

Renowned SE authors and experts acknowledge the plurality of the sector and recognise that because organisational goals are self-defined, measures of performance must also be self-defined. Nicholls (2005) identified performance metrics utilised by SE organisations as either qualitative or quantitative. Qualitative models include the balanced scorecard (Kaplan & Norton, 1996; Bull, 2007), the triple bottom line (Elkington, 2001), the social audit (Zadek, 1998), and the family of measures (Sawhill & Williamson, 2001). Until present, the only recognized quantitative model for performance measurement among SEs is the Social Return on Investment (SROI) method (NEF, 2004; REDF, 2000). A review of the strengths and weaknesses of each performance metric is provided in this section.

Balanced scorecard

Pioneered by Kaplan and Norton (1996), the BSC is a holistic performance management tool that is intended to support the pursuance of an organisation's mission by focusing management attention on controlling the performance of internal activities perceived as causally related to achieving the stated goals or objectives derived from it. The corporate model of the BSC includes a financial perspective, customer perspective, internal process perspective, and a learning and growth perspective. Understandably, the model used in the for-profit world put emphasis on financial performance among all other domains. The non-profit model of the BSC amended Kaplan and Norton's original model to match the needs of SEs. Bull's (2007) model accommodated the social and environmental returns into the financial perspective and reduced the predominance of the financial performance to provide a more balanced weight of all domains. A strength of the BSC model is that it has the capability to establish causal links between non-financial measures to overall mission success. The BSC can ensure that SE managers make decisions that are strategy-led rather than reactions to short-term conditions in the marketplace. However, in common with the other largely qualitative metrics, it lacks a comparative dimension and therefore is of most value in internal decision making rather than demonstrating public accountability (Wei-Skillern, 2007; Kramer, 2005).

Social audit

Social auditing reports are basically longitudinal assessments of an organisation's internal performance using descriptive metrics. It enables organizations to evaluate and demonstrate the social, environmental, and economic benefits as well as limitations of its projects. In this manner, organisations can measure the degree to which it has fulfilled its missions and objectives. This performance metric is adapted from Zadek's (1998) principles of social accountability. Social auditing focuses on measuring the impact of organisational performance particularly on how it has met non-financial objectives through a systematised and regular monitoring of its internal performance and stakeholder views and opinions.

This performance metric is strong in providing the organisation with a thorough narrative of the specific objectives and activities that the organisation can utilize to demonstrate how it has progressed over time. The impact of projects is described in "social accounts" which map progress measured against strategic objectives. An example would be to track down 1) the number of people who gained employment or 2) the number of beneficiaries of health care programs.

A weakness of the social audit is that it fails to quantify how an organisation has actually accomplished its social purpose. Like the balanced scorecard, it lacks comparative value and its nature as a marketing strategy for stakeholders make it a poor measure of performance for organisations.

Triple bottom line

The simplest of the qualitative social metrics is the triple bottom line (Elkington, 1997, 2001). This model requires an enterprise's accounting system to incorporate not only the traditional measures of financial performance, but also social and environmental outcomes. However, unlike financial accounts, the social and environmental audits are typically descriptive, rather than quantitative, and partial and subjective rather than complete and objective. Any external comparative dimension is also typically lacking (although internal, longitudinal comparison is possible). This is primarily the consequence of the lack of agreed social and environmental performance benchmarks. Finally, in this model, the three bottom lines are not weighted or integrated into any final statement of performance. The triple bottom line is very useful for conventional businesses, as it reminds them to consider the social and environmental outcomes of their commercial behaviours and audit their progress in all three areas. However, for a social venture the model has little value. Since such activity is intrinsically concerned with generating social and environmental outcomes, as well as commercial returns, there is little need to encourage an acknowledgement of the role of all three in assessing business performance.

Family of Measures

As a reaction to the perceived lack of focus of existing social metrics, Sawhill and Williamson (2001) developed the Family of Measures model to better articulate a venture's progress towards its mission objectives. As they commented (p.98), 'Every nonprofit organization should measure its progress towards fulfilling its mission, its success in mobilizing its resources, and its staff's effectiveness on the job'. The key point they made was that not-for-profit organizations needed to devise metrics that reflect the detail of their mission objectives, rather than their organizational performance. Thus, a social venture would need to focus less on its overall level of sales growth or profitability and more on the social impact it is achieving. The model was built around three sets of linked metrics: impact measures; activity measures; capacity measures. Impact measures aim to map the progress towards fulfilling the organization's mission and the long-term objectives that drive it. For these to work, the elements of the mission must be broken down into quantifiable and specific objectives. It may be that detailed research is required for this to be effective. Activity measures chart the progress towards goals and programme implementation that drive organizational behaviour. These would include the success or otherwise of mobilising resources toward programme implementations. Capacity measures capture the effectiveness and progress of all levels of the organization that keep it operational. These might include resource allocation and fund raising or market share.

Social Return on Investment

To date the only rigorously quantitative model of social impact measurement is the Social Return on Investment (SROI) framework devised by Emerson and the Roberts Enterprise Development Fund (Emerson, 1999, 2003; REDF, 2000) and, more recently, extended by the New Economics Foundation (NEF, 2004). Social Return on Investment (SROI) is a measure that captures the value of social benefits. It is a relatively new measure developed by Jed Emerson and the Roberts Enterprise Development Fund (REDF). SROI represents a development of traditional cost-benefit analysis as a way of translating some of the social objectives of organisations into financial measures (generally gains or losses to public expenditure) (Aeron -Thomas et al 2004).

It represents the most sophisticated attempt to measure the impact of social ventures that is currently available. NEF comments: Social Return on Investment mirrors the standard financial measure of economic return but shows how organizations of all kinds create value beyond the economic. This is particularly true for those organizations in the social economy that may search for either economic and social value, or just social value. When compared to mainstream businesses, they may or may not achieve similar levels of financial return, but even if they do not, the value to society of the social or environmental returns that they create may well be equal or higher (2004: p.3)

SROI methodology allows social ventures to answer the following questions more effectively:

How do we know whether we are accomplishing what we set out to do?

How can we make informed decisions about the ongoing use of resources?

How can we test and convince others of what we have achieved and show the value of our social outputs in terms that would be understood by those financing social ventures? (NEF, 2004: p.6)

SROI develops a quantitative approach that uses the fundamentals of Cost Benefit Analysis reporting on return on investment as its starting point. This approach is very different from the more qualitative approach of Social Accounting, for example. SROI aims to guide future investment decisions and justify past decision-making, whereas Social Accounting demonstrates accountability and generates stakeholder engagement. Both models are clearly useful and, in best practice, should be used in tandem.

The SROI model is in three parts: the first calculates the full blended value of a social venture (combing its enterprise value and social purpose value); the second establishes the investment in the project; the third then calculates the blended return on investment (combining the enterprise and social returns).

Methods of measurement

The key is to measure the right things (Sawhill & Williamson, 2001, p. 100).

Strive for SMART measures: Specific, Measurable, Accountable, Results-oriented, and Time-bound.

Specific: the performance measure has to indicate exactly what result is expected so that the performance can be judged accurately. The specificity of the measure is aided by clear definitions and standards for data collection, standardization, and reporting across departments and among employees involved in the use of measurement.

Measurable: the intended result has to be someting that can be measured and reported in quantitative and/or clear qualitative terms. This characteristic is achieved when programs set numeric targets or employ an evaluative approach that can ascetain in a definitive manner whether performance expectations have been met.

Accountable: the performance measure has to be "owned" by a specific business line or employee base to the degree that someone, or some group, is held accountable for the performance measure to ensure that the results are indeed produced. Accountability is more than clarifying who is charged with achieving the result; it requires that management has devised targets based on what reasonably can be produced by the program during a given period of time. Accountability cannot be achieved if targets are unreasonable from the start.

Results-oriented: the performance measure must be aligned to the outcome and track an important value or benefit needed to advance the strategies and achieve the ned results of the organization: mission impact. A performance measurement meets this test if it (1) measures an end or intermediate outcome or (2) links to a process or activity that, if successful, will achieve an outcome.

Time-bound: the performance measure must set a specific time frame for the results to be produced as well as allow for the reporting of performance in a timely manner. In this case, the organization must have measures to provide fresh enough data to be used by management for adjustments in the program and corrective action if necessary.

The first key question is, however: what to measure? There are a variety of possible social impacts to consider. The simplest are quantifiable outputs, such as the volume of sales of generated by a social venture or the number of people employed. However, these figures alone are of limited value. Their main contribution is in longitudinal tracking of organizational performance over time. More useful is data on the social outcomes of such ventures, including the generation of social value added impacts. These data might include, for example, not only a measure of the improvement in an individual target stakeholder's income or welfare, but also the improvement in his or her family's standard of living or the overall well-being of his or her community. The public sector value should also be factored in. This may involve welfare cost savings by reducing reliance on social service support or, in the case of employment projects,

increased government revenue through higher tax revenues. Furthermore, a full appreciation of social value added should attempt to capture other positive externalities such as psychological or developmental benefits and their wider role across communities (often outside of the target group).

The second key question is: how will data be measured? Attempting to measure individual human impacts is problematic, since they will inevitably be highly personal and individualized and may include multiple unique variables. It is likely, therefore, that aggregated data will often be used such that some of the nuances of each case will be lost. Moreover, the most appropriate timescales for social impact measurement are also often unclear. Many of the impacts of social ventures will be almost immediate (such as improved income levels or health benefits) others (such as greater selfconfidence, improved education or community building) may only pay off after several years.