Analysis of the London Ambulance Service Computer Aided Dispatch system failure

This paper will analyze one of the most prominent computerized system failures in the past 10 years- the failure of the London Ambulance Service Computer Aided Dispatch system-hereafter referred to as LASCAD. Unlike the common one dimensional explanations for system failure that view Information systems as mainly a neutral technical artifact ( Klein and Hirscheim, 1987), this paper will attempt to explore the more multi-faceted nature of systems failure which is closer to the reality that system exist in. This analysis will be anchored in the concepts of holism and emergent properties as described by Francis and Roland Bee (2005), Managing Information and Statistics, 2005, whereby the approach taken to analysis emphasizes the system relationships and processes and results of its interactions. References will be made to existing frameworks used to investigate system failure in particular the Sauer model Sauer (1993). Details of the description of the system and the failure will be drawn mainly from a paper on Information System failure and risk Assessment by Paul Beynon Davis (Computer studies technical report University of Glamorgan, 1994b).From this investigation existing methods of preventing or solving software systems failure will be explored in the context of the LASCAD system to look into recommendations and lessons learnt to prevent such failures .This will particularly focus on risk handling as proposed by B.W Boehm ( 1991) and the Goal Question Metrics by Solingen and E. Berghout (1999).

Summary of the LASCAD System Failure Case Study

The LASCAD system was a computer aided ambulance dispatch system established at the head quarters of the London Ambulance Service. According to Page et al (1993), the expected functions of the system are described below:

Call taking: Acceptance of calls and incident details

Resource Identification: Particularly which ambulance to send to an incident

Resource Mobilization: Communicating details of an incident to the appropriate ambulance

Resource Management: The positioning of suitably equipped and staffed vehicles to minimize response times

Management Information: This involves the collation information to assess performance, resource management and planning.

This system was supposed to solve the problems related to manual dispatch systems including time consuming and error prone identification of the precise incident location, paperwork and maintenance of current vehicle status information.

The LASCAD system objective was to automate these manual human intensive tasks by using an events based and ruled based approach and integrating a Geographical Information System (GIS) to provide location details. In this system the callers, incident and patient details would be recorded and transmitted to the dispatchers. Through the use of radio signals and GIS the system is able to determine the ambulance nearest to the patient. After dispatch the ambulance crew was expected to acknowledge the dispatch message and the system would then detect whether the ambulance was headed in the right direction. Finally the system would alert the controller on the ambulances arrival to the scene, hospital and when it becomes free again.

Figure1: LASCAD flow chart (Paul Beynon Davis, 1994)

This explanation of expectations of the systems functionality is pretty linear and even simplistic but on closer examination one is able to construe the complexities that are involved in delivering such expectations. This will become more apparent in the following section highlighting the system failure and later on the events leading to the failure.

Between 26th and 27th October 1992 (Paul Beynon Davis, 1994), the system started to fail. It was reported that as a result of a flood of emergency calls bogged down the system and this resulted in erratic behavior of the system involving calls being wiped off the screen and automatic alerts indicating unacknowledged calls to ambulances.

According to the Guardian newspaper, 1992, it was claimed that 20-30 people may have lost their lives due to ambulance delays. Indeed the impact of this failure was tremendous and as expected triggered various responses as to what was the cause of the failure. According to Donaldson and Jenkins (2000) in their paper on System Failures: An Approach to understanding what can go wrong, the causes of system failure are complex and interact with each other and in some cases a single factor may single out the problem while in others a combination of many small and apparently insignificant factors are to blame. This merely says that it is difficult to analyze causes of systems failure which would only be closely understood through multi cause analysis stemming from the soft systems methodology. It also becomes apparent that everything is not always as it seems, a good example is the Arriane V rocket (ESA Press release Nr 33-96-July 1996) which failed courtesy of its navigation software being inappropriate for the rockets design. This was not actually a software failure as may have been though in the outset but a problem with the overall incorrect assembly of the rocket. As it were the software performed to its specification. This is akin to expectation failure which Lyytinen and Hirscheim (1987) describe as the inability of an IS system to meet specific stakeholder groups expectations, they signify a gap between an existing situation and a desired situation for members of a particular stakeholder group.

This is further enhanced by Donaldson and Jenkins (2000) system failure analysis detailing high public expectancy of computer technology, Fashion/popularity of systems obscuring its basic objectives and the varying stakeholder interests creating different perceptions of the system.

Analysis of the LASCAD System failure

Following the above outline of the system failure and prelude of expected challenges in analyzing system failure this section will attempt to shed detailed insight into the failure. The analysis will follow Sauer’s model (Sauer, 1993), of investigating system failure which is based on a triangle of dependencies between:

Project organization

Information system

Supporters

The multi-faceted nature of systems failure alluded to in the introduction would mean that even this triangle is not a closed system but is affected by other contextual factors of which according to Sauer consists of cognitive limits-(e.g. limits of communication), technical process-( constraints from structured nature of computerized systems or development methodology), environment-(constraints by customers, suppliers, competitors, regulators), Politics, internal project structure and history associated with previous information system projects.

Project organization

In light of the LASCAD project failure the project organization from inception is very wanting. Firstly following a public inquiry report on the failure (Page et al, 1993), it is claimed that the London Ambulance Service (LAS) management put price before quality and committed to an over ambitious project timetable. This was evidenced by the selection of a supplier who has no experience in building ambulance dispatch systems but had significantly underbid a more established supplier. This was made worse by the management putting the supplier under immense pressure to deliver the system quickly.

Secondly the project management team did not follow the PRINCE (Projects in Controlled Environments) project management method prescribed for public sector projects.

Thirdly it was found that the system was incomplete and unstable and particularly the emergency backup system was untested. This was further compounded by inconsistent and incomplete user training.

Information system

In terms of the information system dimension the report of the public inquiry (Page et al, 1993) suggests that the failure was not a result of technical issues since on overall the system did what it was designed to do. It goes further to explain that at the onset the loads on the system were light and the control staff could easily cope with various problems associated with ambulance crews pressing wrong buttons, radio black spots, communication hand-shaking problems etc. When these incidents increased incorrect vehicle location and status information received by the controllers also increased resulting in the failure to cope with the load leading to fewer resources to allocate to incidents and subsequent delays in response times.

Supporters/stakeholders

As defined by Paul Beynon Davis (1995), supporters/stakeholders defined as people sharing a pool of values that define what the desirable features of an information system and how they should be obtained. The stakeholders have different views and expectations of the system of which such a mismatch in perceptions in this case contributed to the failure. This is depicted below:

Figure 2: LASCAD system perceptions rich picture

LAS Management:

The London Ambulance Service (LAS) management viewed the system as a way to improve service to patients by putting in place mechanisms that would ensure objective and impartial resource mobilization through automation. The LAS management was also influenced by a past experience involving a failed computerized dispatch system project and pressure from organization-wide restructuring that put them under immense pressure to succeed

Control room staff:

The staff in the control room found the system to be too complicated and did not trust the motives behind implementing a computerized system

Ambulance staff:

The ambulance crews were more comfortable with the radio call systems that they had been used to and did not have confidence in the new system as they did not see the need for it and found it too complicated

Union:

The staff union found that there were no requisite consultations done before making the decision to acquire the system and as such the already strained relationship between management and staff was worsened.

Hardware and software suppliers:

The system suppliers were not sure how to implement the system in the first place and this was compounded by tight deadlines from what they thought to be a disorganized client.

Related to these perspectives are contextual factors concerning political environment courtesy of the overarching influence of the National Health Service (NHS) on the London Ambulance Service which is the LAS oversight body (Beynon-Davis 1994).The NHS is characterized by the lack of a unitary power structure and is made up of a network of different health organizations. The implication on a new information system is a very careful political balance in the impact the impact the system will have on the relationships in this network (Checkland and Scholes, 1990).

As posit by page et al, (1993), the LASCAD project was greatly affected by internal tensions in within the NHS which had commissioned major reforms in the London Ambulance Service including restructuring that resulted in the reduction of middle management from 263 to 53. It is clear that this resulted in strained relationships and an environment of mistrust and obtrusiveness when it came to any changes, which affected the LASCAD project.

So far what is clear is the multifaceted nature of the failure that results from various causes of the failure that is common in computerized information systems, which Paul Beynon-Davis describe as web-like in nature. It has been reported that 92% of all system failures involved failures of technical interaction with cognitive /organizational factors (Mackenzie, 1994).

This as it were it is essential to trace the true causes of the system failure. One way of doing this is through multi cause diagrams as mentioned in the section above or Petri nets which use state and event oriented graphs.

The LASCAD project failure is depicted below using a multi cause diagram to explore the events and states on why the failure occurred:

Figure 3: LASCAD system failure multi-cause diagram

Ideas, Recommendations and Lessons Learned

As expressed above using Sauer’s model (Sauer, 1993) of investigating system failure, the dependencies between the project organization, information system and its supporters have come out very clearly. Using the information system dimension the failure is not attributed to technical issues at all, which goes against common place failure attribution of computerized information systems. This begs the question, what constitutes a system failure? Lyytinen and Hirschein (1987) categorize system failure into four:

Correspondence failure: There is a disjoint between the design objectives of the system and what is practically being met by the system.

Process failure: This is characterized by runaway projects that either do not provide a workable system or overrun budgets and time.

Interaction failure: This focuses on utilization of the system i.e. a highly utilized system is considered a success and one that is hardly used is a failure.

Expectation failure: As stated earlier this is the inability of the system to meet a specific stakeholder group’s expectations. The LASCAD system falls into this category as it appears it did not meet various stakeholder group expectations. Donaldson and Jenkins (2000) talk about a 3 dimensional picture where a system totally fails, partially fails or temporarily goes down.

In the case of LASCAD it is taken as a partial failure resulting from a number of flaws that are rectifiable and as such this is not a total failure.

The rectification will mainly involve a reassessment of the entire project taking mainly focusing on the role of risk assessment. Risk is the probability of a negative outcome. Negative outcome is in essence a relative concept as Wilcocks and Margetts (1994) suggest the risk of a negative outcome only becomes a salient problem when the outcome is relevant to stakeholder concerns and interests. Different settings and stakeholders will see different outcomes as salient.

The proposed framework to use in risk assessment follows Wilcocks and Margetts (1994) who put across the following categories to be used in analyzing the development, introduction and use of information systems, these are:

History: Past experiences with information system development.

Outer context: The environment in which the organization is operating e.g. economy, markets, government

Inner context: The characteristics of the organization e.g. structure, strategy

Content: For example project size and difficulty

Processes: For example project management and staffing

Outcomes: Planned and anticipated results.

The proposed risk assessment framework would be implemented through out the development, introduction and use of information systems. This will be used to complement an overarching software management methodology such as the Goal Question Metrics (GQM) mentioned in the introduction and the Capability Maturity Model which outline good practices in project management to ensure project success. In the context of LASCAD the GQM will particularly address the aforementioned failure characteristics in the analysis section through the following stages in development:

Setting specific goals in light of purpose perspective and environment

Refine goals into quantifiable easy to understand questions

Derive requisite metrics and data to answer the questions

There are various methods that can be used in preventing or solving computerized system failure the Capability maturity model and Goal Question Metrics mentioned above are by no means exhaustive nor are they prescriptive. Organizations are different contextually and individual projects also vary in size and complexity and as such would require approaches the methodologies to be customized and scaled for specific organizations and projects. The Capability Maturity model is a prime example that targets improvement in software processes toward a specific target- maturity level that the organization is working toward.

On the other hand there is need to put emphasis on risk management outside of the one dimensional technical orientation to encompass the complexities of computerized systems as seen through the lens of Wilcocks and Margetts (1994) risk management framework.

Conclusion:

The LASCAD system is a good example that portrays the reality of the complex and multi-faceted nature of systems failure. The different perspectives of the system and congruent expectations make even the very definition of the failure unclear. This particular case highlighted the political and social causes of the failure, what has been described as contextual factors. References to various frameworks have been made in the analysis of the failure -Lyytinen and Hirscheim (1987), particularly expectation failure and dependencies in the 3 dimensional Sauer’s model (Sauer, 1993). The failure analysis provided the distillation of the system failure characteristics which describe the true causes of the failure. This was done using rich pictures to accommodate varying perceptions and expectations and multi cause diagrams to explore the various causes of the failure.

Lessons learnt and future remediation of systems failure is centered on risk management and project methodologies ensuring good practice in the development, introduction and use of information systems. As recommended in this paper these should take into consideration contextual/ organizational issues apart from technical aspects of the system.