Performance Measures for (50,000 call per year) EMS

EMS ORGANIZATIONAL PERFORMANCE MEASUREMENT

That the organization implements additional clinical performance measures, including those to evaluate the quality of the EMS.

That the organization uses survey data to evaluate and analyze customer and employee satisfaction and that a proper feedback and control mechanism is in place to use this data to implement required changes.

This report starts from the premise that Emergency Medical Services will be treated as any other service. As a consequence, this type of service reflects the relationship between the service recipient (in this case the patients) and the service provider (in this case the medium-sized organization being analyzed in this report).

This means that this report will use many of the existing research and business literature and apply business principles such as customer and employee satisfaction in presenting and analyzing a comprehensive set of recommended performance measures for a medium-sized (50,000 call per year) emergency medical service organization.

Operational performance measures

The Littleton Fire Rescue service has been providing citizens emergency medical services for decades. However, the performance of their EMS was not measures and its influence on patient outcome could not be determined. The Littleton Fire Rescue department then developed an applied research process intended to identify key performance indicators that could be used in measuring the department’s performance (Zygowicz, 2010).

The operational segment was of particular interest to researchers in this case, as operational performance measures have been identified in order to improve EMS system efficiency and to improve customer service. These measures can be successfully benchmarked by our EMS organization.

One of the measures proposed by the study refers to monitoring individual paramedic performance and giving feedback to crew members. This measure is intended to personalize recommendations for each crew members, which increases their efficiency.

Another important measure in the operational segment is to determine the number of tracheal intubations performed by the staff. If the reported number is lower in comparison with the state average number of tracheal intubations, training programs must be acquired for the organization’s personnel (Zygowitz, 2010).

Measures that should be benchmarked by the organization refer to its communication center or dispatch, light and siren response, and primary EMS complaints. Collecting and analyzing data on patient satisfaction is probably the areas that can provide important information to be used in developing measures that improve satisfaction. However, it is worth mentioning that patent satisfaction surveys have their level of subjectivity. Patients’ responses in such surveys should be measured against more objective indicators in order to build an accurate image of the service’s performance. In order to facilitate the implementation of these surveys, they can be sent to patients together with their transportation bills.

It is recommended that the organization posts on its websites information about performance measures it takes and their indicators. By publishing such results, the organization ensures the transparency of its activity (Zygowitz, 2010).

Clinical performance measures

Myers, Slovis et al. (2007) have conducted a significant amount of research regarding the measures of clinical performance. It is useful to start this subchapter with a brief analysis of some of the measures they have identified and employed in their research. Their study starts with some of the traditional performance measures. One of the most important in this category is the response time intervals.

There are several advantages with regards to the response time intervals. First of all, they are easy to quantify and this is usually very helpful in the dialogue that medical centers have with external shareholders, such as the public and the central administration. Being able to quantify a clinical performance measure will objectively reflect how well such a center is doing.

Benchmarking response times remains a challenge because, as the medical science evolved, more individuals could perform some of the basic resuscitation measures such as advanced life support (ALS) or cardiopulmonary resuscitation (CPR). The latter is now receiving less attention as part of the measuring process and the ALS response-time interval tends to be the norm in clinical performance measurement. It shows how soon it takes the responders to arrive and perform ALS.

The average response time is already in use in this organization. The only problem is that this is the only performance measure that the organization uses. However, it is a useful measure in regulating the organization’s human resources, particularly in terms of the number of paramedics it uses and how they should be allocated to the different cases.

Among other traditional performance measurements that are mentioned in this study are the out of hospital cardiac arrest survival rates. The complaints that Myers et al. have is that this type of measurement approach it is only focused on a particular area of EMS action, namely cardiac arrest. According to their work, this only covers about 2% of the total number of EMS responses. Other approaches, such as the ST-Segment Elevation Myocardial Infarction (STEMI) Performance Measures or Respiratory Distress Performance Measures appear to have a similar problem.

The model proposed by Myers et al. For clinical performance benchmarking is applied, as the case study mentioned, to large urban or suburban EMS Systems. The organization we are discussing here is a medium-sized EMS provider. As such, the best approach in terms of clinical performance measures should be to focus on the general measures and implement a series of quality performance measurement systems, such as the ones described further below.

As El Sayed (2011) pointed out, more and more emergency medical services have started to be evaluated in a similar manner with healthcare services, where an important aspect is the quality of these services. In the same manner as the healthcare industry, measurements of quality in EMS have looked to quality management systems used in the business industry (El Sayed, 2011), since, after all, the objective of this process is to improve the quality of service for the end consumer (or the quality of the product).

An important category of measures used in this area is process data. Process data looks at the interaction between the patient and the prehospital or EMS provider and analyzes the different steps that are part of this interaction. This is a very useful approach, because it involves a very direct evaluation of the quality of EMS. However, the more complex the interaction (and, at the same time, the more complex the level of services that the organization provides to its patients), the more difficult this analysis process is likely to become.

In the case of this particular organization, it seems that its medium size (50,000 calls a year) encourages such an approach, since it is sufficiently small to ensure a proper implementation, without too many resources. Likely, the best way forward would be to implement a separate structure that would monitor, record and correct the quality of services towards patients.

Another type of quality performance measurement that El Sayed discussed is outcome data. This type of quality measurement looks at the change in the state of the patient following EMS. El Sayed points out to several important advantages and disadvantages for this type of approach. First of all, it is very easy to implement and understand by evaluators. After all, it involves a transparent analysis of whether the condition of the patient has improved during EMS or not.

Second, it is a comprehensive approach to evaluating the quality of EMS. In order to evaluate the outcome, it looks at all different aspects about EMS and it will judge the effectiveness of an intervention from a wider perspective than the process data approach. Rather than sequential, such as the process data approach, outcome research is more holistic and, as a consequence, the results will probably better reflect the situation on the ground.

There are also a few disadvantages with this approach to quality measurement. El Sayed points out that theory is yet underdeveloped when it comes to standardizing norms related to outcome. For some medical facilities, improvement in a condition may be a show of quality care, while for others, this may just be the regular standard or may just be a consequence of other factors than medical care.

However, this is an area that is gradually improving and concerted efforts are undertaken in this sense. For example, the U.S. National Highway Traffic Safety Administration (NHTSA) has launched the EMS Outcomes Project. The concrete result of this project was to define six categories of potential outcomes, ranging from survival to limited disability and to cost-effectiveness.

In the particular case of this organization, the outcome model is an excellent, low-cost solution, but it implies a customized research in order to determine how the model could be best matched on the existing structure and on the organization’s characteristics. Some of the categories of potential outcomes that the NHTSA has put forward can be retained as such.

Finally, another category of measures for the quality of services is structural data. This approach looks at the different components in the EMS system and evaluates each of these depending on its capacities. There is one significant disadvantage with this approach. There is little link between this and the final outcome. The final outcome should link, somehow, the physical components with the patient, the customer and final recipient of the EMS.

With this in mind, our organization should implement minimal components of this category of measurement tools for quality. Certainly, one needs to be able to continuously evaluate the physical components of the EMS systems, but this should be minimal and should just look at how reasonable the components are functioning, as comparative to the established benchmark in the field.

Resuming this section of the report, from a quality measurement perspective, after having analyzed the three categories of measurement approaches that El Sayed describes in his work, the outcome model should be favored in our case, given the low-cost and overarching, holistic approach. At the same time, parts of the other two categories should also be included, so as to measure particular areas of interest for a complete evaluation of the quality of EMS systems.

System Design and Structure

Before measuring the performance of financial, technical and human resources it is important to pay attention to the design and structure of the EMS systems. These are the primordial factors that support other resources’ efficiency. In other words, all resources’ performance relies on how appropriate the system has been designed and structured.

One of the measures proposed by the NHTSA (2009) refers to determining the type of Emergency Medical Dispatch Protocol Reference System (EMPRDS) used by the EMS dispatch center. The objective of this measure is to determine Ems dispatch centers to use APCO, Medical Priority Dispatch System, power phone, or another type of system that can facilitate and improve the efficiency of the center in comparison to not using any such system.

One system design and structure measure that requires minimum effort to implement and can contribute to improving EMS performance refers to how the EMPRDS is connected with the lights and siren use response mode. In other words, it refers to using services or systems that rely on EMDPRS in order to determine the use of lights and siren when responding to calls. If the percentage of users of such systems would increase, it is likely that the effect would be similar to EMS providers’ performance.

In addition to these measures, EMS performance can also be improved by connecting the responder level dispatch to the EMDPRS it uses. This is intended to determine the level of care capability of the dispatched responders. This measure should be addressed on an annual basis in order to determine improvements.

As one can observe, the organization’s performance relies on the EMPRDS its dispatch uses, as each category of systems have their advantages and their disadvantages. The system’s connectedness to the lights and siren use is another factor that can be addressed in order to improve patient satisfaction. These measures should also be backed by connecting the EMPRDS with the responder level dispatch.

Human Resources (culture, training, safety, credentialing, etc.)

With personnel’s efficiency playing such a significantly important role in the performance of EMS, human resources performance measures have been included by the NHTSA (2009) in its project. The performance measure proposed by the administration refers to identifying the turnover rate for EMS providers. This measure is requested by the need for experienced personnel in each area served through EMS. Its objective is to reduce turnover rate.

It is recommended to measure the turnover rate for EMS providers on an annual basis. Another aspect regarding this measure that must be brought to attention is the fact that the NNHTSA proposes it as an interim performance measure to be applied until standardization of licensure and certification levels is achieved. In order to reach this objective, EMS providers turnover rate can be calculated by comparing the number of identified licenses on each level to those of the previous year. Given the differences between licenses and certification, no other standard method can be applied.

Human resources have a significant impact on EMS performance and on patient satisfaction and their activity should be assessed and improved through training programs. Such programs should be oriented towards specific clinical care areas, but also towards communication with staff and with patients.

Clinical Care and Outcome

Some of the most important measures that can have a significant impact on the quality of service provided to patients refer to the clinical care and the outcome derived from EMS providers’ activity. In order to address this line of work with the best quality of response it is important to measure the mean time and the 90th percentile time from PSAP contact to the initial defibrillation in cases of cardiac arrest taking place before EMS arrive at the location where defibrillation is attempted (NHTSA, 2009).

This measure is recommended by patients’ need for speed of access to defibrillation. The objective in this case is to reduce the time associated with this process.

Another measure that can be applied in situations where cardiac arrest takes place before EMS providers arrive refers to identifying the mean time and the 90th percentile time from PSAP contact to the initial analysis of rhythm in cases where an EKG is obtained. This measure is required by patients’ need for speed of access to rhythm analysis. The objective of this measure refers to reducing the time associated with this process.

These measures are to be complete by others proposed by NHTSA, like determining the percentage of patients who meet 2006 CDC/ACS field triage criteria for transfer to trauma center are transported to the trauma center. The objective is to increase rate in patients that need transportation to trauma centers.

Patients’ pain level is an area that requires its own performance measure. This objective can be reached by determining the percentage of patients that reported decreased pain, increased pain, or no change in pain, by comparing first and last scale pain values. Following this measure, the organization should also determine the percentage of patients with a reported pain of 7 or above on a scale between 0 and 10 have benefitted from subsequent interventions associated with pain relief. It is expected that increasing this percentage can determine a performance increase for EMS providers.

This line of measures is recommended to be continued by determining the percentage of patients over 35 years of age with suspected cardiac chest pain that received a 12-lead ECG. The patients’ need that determines this measure is represented by the early performance of diagnostic EKG that can accelerate definitive care. For the same category of patients it also seems important to determine the percentage that received an aspirin.

EMS providers’ performance can also be measured by determining the percentage of patients with 12 lead ECG indicated ST elevation myocardial infarction was transported to a hospital that was able to provide emergency intervention cardiac catheterization.

Clinical Care measures should also refer to patients experiencing cardiac arrests after EMS arrival. The measure in such case is to determine the percentage in this patients group that survives to discharge from the emergency department and discharge from the hospital. This measure indicates successful resuscitation from cardiac arrest. The objective is to increase rates.

The clinical care and outcome segment is one of the most important to be addressed by performance measures. The numerous areas in the field that can must improved can be summarized by measuring quality of response, improving field triage criteria, accurate assessment of patients’ pain level, and other measures referring to patients with cardiac arrest. However, it is worth noticing that clinical care and outcome measures are linked with operational measures. In other words, the efficiency of clinical activity depends on the performance of the operational segment.

Response

The response time to patients’ needs is one of the most important areas of EMS that require thorough performance measures. This is the area where the organization has already implemented such a measure, regarding the time it takes EMS providers to respond to calls they receive from patients. However, this measure should be backed by others that can provide complex and accurate information that can determine improvements in these services. Additional measures refer to determining the mean and 90th percentile emergency scene time intervals. This impacts timely triage, treatment, and departure from scene. The objective is to reduce this indicator. Another similar indicator that must be measured refers to determining the mean and 90th percentile emergency transport time intervals.

Measures on response time should be paid specific attention, as efficiency in this area has a significant impact on other indicators of the organization, but also oh trauma centers and hospitals.

Finance/Funding

There is a strong connection between performance and financial resources. Significant investments are required in order to benefit from modern equipment that can improve the quality of services EMS providers offer patients. In addition to this, reducing the costs associated with these services where possible can allow such investments.

Therefore, in order to identify strategies that can improve the financial aspects, it is important to identify the total EMS cost per capita. This measure ensures cost of service awareness and comparison (NHTSA, 2009). Given the nature of expenses, this cost should be measured on an annual basis.

Quality Management

The quality of EMS service providers can be measured by a series of indicators, but it is patients’ satisfaction level that offers the most accurate point-of-view on the subject. A measure that should be addressed by the organization in order to assess quality levels refers to identifying the percentage of patients that are satisfied with their EMS experience with the organization. The objective is to obviously increase satisfaction levels. However, useful information would be derived by identifying the reasons that determine patients’ level of satisfaction. The accuracy of this measure is influenced by another measure referring to determining the percentage of patients the organization surveys in order to measure patients’ satisfaction levels.

Measures that refer to more objective indicators are: determining the percentage of patients in respiratory arrest or distress that receive oxygen, determining the rate of undetected esophageal intubations, determining the rate of EMS crashes per 1,000 responses, determining the rate of EMS crashes per 100,000 fleet miles, determining the rate of injuries because of EMS crashes. These are measures that address specific activity within the EMS and they’re necessary in identifying service quality measures.

Community and Demographics

The area of community and demographics that can be improved by actions taken by the EMS organization is call complaints distribution and rate. The measure in reaching this objective would refer to determining the number and distribution or primary complaints the organization responds to. In order to improve personnel’s approach to such calls, it is recommended to invest in training programs and to carefully plan the activity so that employees improve their performance with the different call types they experience.

Financial

Financial feasibility is one of the most important objectives that the EMS organization can establish on an ongoing basis. This reflects its current performance levels, but also potential for future improvements. Financial planning must be carefully linked with the needs of each season: winter seasons where influenza tends to outburst and summer seasons where the excessively hot temperature affects an increased number of patients require additional resources.

Similar EMS organizations have determined that they must invest in purchasing four ambulances to help them improve response and quality of service to patients. However, as this would be an important purchase that might affect financial indicators, it is recommended to purchase two ambulances before the 2014 influenza season, while the other two can be purchased in Q4 2015. The cost of these ambulances is likely to e around $900,000. As a performance measure resulted from this acquisition, the cost per hour of ambulance service should be determined on an annual basis.

Customer and Employee Satisfaction/Engagement

Customer satisfaction survey data is often used in order to measure customer satisfaction and can also be translated for employee satisfaction. Balridge (2002), among others, has emphasized the importance of using survey data as a direct mechanism for feedback and subsequent control and improvement. His model describes the intrinsic link between customer feedback (obtained with survey data), performance measurement and subsequent action to correct and improve performance.

This discussion around customer satisfaction, communication with the customer and undertaking the necessary measures in order to correct the customers’ observations has long been discussed in business literature. Its application in EMS comes as a logical translation, given the fact that, as described in the introduction, EMS are similar to any other service-providing industries and rely on similar approaches to measuring performance and competitiveness, including setting benchmarks and obtaining customer feedback.

It is also interesting to note, looking at what matters for customers, as shown by The Customer Communicator (2004), that EMS does have its particularities. Some of the most important elements that customers use when judging the performance of an organization are timely delivery and the quality of services. This is similar with EMS, however, one may judge that, in critical cases, the importance of timely delivery (in this case, the timely arrival to the scene) becomes more important.

Bruce (2004) proposes the application of survey data to measure EMS customer satisfaction and, as a consequence, to improve service delivery. His case in point is the Rialto Fire Department in California. The Rialto Fire Department was collecting customer survey data, but it was not using them in any way. As such, while it had potentially recognized the important of having regular feedback from its customers and beneficiaries of services and while it had started the first phase, this is where things had stopped.

The implementation of a proper survey data management system, including with short, medium and long-term goal, affected the organization in different directions, including by setting clear objectives as to how the organization could improve its performance and by identifying new areas where additional survey data could be obtained in order to improve performance measurement.

In a similar manner, survey data can be successfully used to measure employee satisfaction. First, one needs to address why employee satisfaction is such an important component of the organization’s well-doing. It is the employees who perform the everyday actions and their satisfaction will ensure that their performance is also maximized.

At the same time, employee satisfaction maximizes employee retention, which will lower overall costs for the organization. These costs would be associated with human resource activities such as recruitment and selection, as well as the actual hiring process. Further more, higher employee retention will allow the organization to invest in training programs and other instruments of personnel development.

As mentioned in the recommendations, the use of survey data by this organization to measure and, subsequently, react in consequence, the performance and the customer satisfaction is highly advisable. A solution across the board is proposed, whereby all customers that have received emergency medical services are asked to fill in a questionnaire (or their family members), a different questionnaire applying in the case of the employees.

Conclusions

The report focuses on two main areas: operational and clinical performance areas. Literature in the field, as well as research studies developed by several emergency medical services providers have been analyzed in order develop a comprehensive report presenting performance measures for improving system design and structure, human resources, clinical care and outcome, response, finance and funding, quality management, community demographics, financial performance, customer and employee satisfaction.

The importance of such reports relies on responding to the need of developing an accurate set of indicators that can assess emergency medical services’ actual performance. Evidence-based indicators that are used in most cases are rather good in measuring these services’ effectiveness, but they are limited by presenting only one side of the situation.

In accordance with the stakeholder theory, each category of stakeholders has a different objective to reach by the activity of the EMS. Therefore, each stakeholder’s their perspective on its performance is different in comparison with that of other stakeholders. In order to make decisions and develop successful strategies for improving EMS activity, managers must rely on a complex perspective that encompasses recommendations from all stakeholders. In order to reach this objective, employee and patient survey should be administered. In addition to this, managers can order internal audits that evaluate the organization’s activity.

The balanced score card is a successful system that can be applied in the case of EMS organizations also. As examples of whole system measures that can be included in a balanced score card system El Sayed recommends patient satisfaction with care score, rate of adverse events, incidence of occupational injuries and illnesses, and health care cost per capita. However, the success of implementing these measures relies also on communicating them across all levels of the EMS organization.

Multifactorial models of EMS system performance determine complex solutions due to their expended documentation. This includes care for ST-segment elevation, MI, pulmonary edema, bronchospasm, seizure, and trauma patients.

The importance of EMS systems evaluation models is that it allows to extrapolate information that can be used in saving an increased number of lives. Literature in the field also provides similar information that if studied against certain indicators can lead to such situations. However, such models also have their limitations. Much of their limitation can be attributed to the fact that the number of high quality trials for infrequently occurring conditions does not suffice for drawing strong conclusions and applying them in the practice of EMS.

This report has also raised the question of what quality of care really means. The answer to this question is offered by the Institute of Medicine that has identified safe, timely, effective, efficient, equitable and patient centered as domains of quality of care (Welch et al., 2010). The Institute’s specialists also advise that the timelines of emergency care has a significant impact on its quality. Therefore, the improvement of EMS operations is strongly required by internal factors, but also by external ones.

Specialists also point out an important factor that influences comparisons between EMS studies. They warn about developing a common language that all EMS stakeholders can speak in order to ensure consensus. In order to reach this objective, it is important to develop common definitions of key terms, timestamps and metrics. Otherwise, the tracking, measuring and communication processes can be affected (Welch et al., 2010).

The challenge of measuring performance in the emergency medical services systems can become a real challenge unless properly structured. It is important to identify all types of performance indicators that address both the clinical and operational segments, the benefits they provide to research, but also the shortcomings associated with each of them. In addition to this, EMS organizations are recommended to address evidence-based bundles in order to ensure they provide high quality care to their patients.

One of the recommendations in this report refers to developing customer satisfaction surveys in order to identify patients’ opinion and assessment of the services they benefit from the EMS organization. The subjectivity of the information identified through such surveys should be counteracted by the analysis of several indicators of the clinical and operational activity of the organization.

However, the important fact that must be mentioned in this case is that information provided by the surveys should be applied to service improvement. If the organization limits itself to investigating customer satisfaction, but does not use the information for improving its services, this means not much progress is made.

For best results, the measured performance levels should be compared against previous levels, but also against a clear set of objectives. Developing sets of goals and targets draws the direction the EMS organization must follow. Comparing against other EMS organizations can also be useful if they are similar in size to a 50,000 calls per year unit. State average indicators can also be used for comparison of performance.

Bibliography

1. Balridge National Quality Program (2002). Criteria for performance excellence. Gaithesburg: National Institute for Standards and Technology

2. The Customer Communicator (TCC) (2005). Alexander Communications Group, 28 (1) 2.

3. National Highway Traffic Safety Administration (2009). Emergency Medical Services: Performance Measures, Recommended Attributes and Indicators for System and Service performance.

4. Bruce, J. (2004). Application of EMS Customer Satisfaction Survey Data to Improve Service Delivery at Rialto Fire Department.

5. El Sayed, M. (2012). Measuring Quality in Emergency Medical Services: A Review of Clinical Performance Indicators. Emergency Medicine International,

6. Slovis, C. et al. (2007). Evidence-Based Performance Measures for Emergency Medical Services Systems: A Model for Expanded EMS Benchmarking. The 2007 Consortium U.S. Metropolitan Municipalities’ EMS Medical Directors.

7. Zygowitz, W. (2010). Developing Key Performance Indicators to Improve Patient Care and Outcome at Littleton Fire Rescue.

8. Welch, S. et al. (2010). Emergency Department Operational Metrics, Measures and Definitions: Results of the Second Performance Measures and Benchmarking Summit. American College of Emergency Physicians.