Table of contents
“Without data, you’re just a person with an opinion”
W. Edwards Deming
Introduction
Key performance indicators (KPIs) play a vital role in medical technology companies that have implemented a quality management system in accordance with ISO 13485. KPIs are used to monitor the effectiveness of processes and ensure compliance with regulatory requirements. ISO 13485 requires organizations to use suitable methods to monitor and measure QMS processes, taking corrective action if planned results are not achieved.
KPIs act as a ‘health check’ for processes, providing objective evidence of how well targets have been met in management reviews and audits. However, KPIs should be a means of control and improvement, not an end in themselves. In other words, KPIs should help management and process owners make well-informed decisions and identify weaknesses, rather than simply producing columns of figures.
In this context, quality management (QM) acts as a supporting function for management: it provides analyses and initiates optimization but does not operate independently of the operational business. Each KPI must be embedded in the corporate context and contribute to overarching management, justifying the considerable maintenance effort of a KPI system through increased quality and efficiency.
KPIs as control variables in process controlling
KPIs are often referred to as “scorecards” or control variables. Like gauges in a cockpit, KPIs indicate whether your processes and company are on track. KPIs support management in planning, monitoring, and improving processes.
It is important to understand that KPIs are integrated into an overarching management system and are part of a control loop.
The classic PDCA (Plan-Do-Check-Act) model illustrates this cycle. First, goals are planned, then measures are implemented. Next, the results are checked using key figures, and then optimizations are introduced based on the findings. KPIs are especially essential in the check and act steps of this cycle. To make target/actual deviations in processes visible and provide effective control impulses, parameters are needed to measure or quantify process quality. The following always applies: KPIs are only useful if they are used for control and decision-making. A low or high KPI alone does not improve anything — it is the interpretation and derivation of improvement measures that make the difference. Therefore, quality management should work closely with process owners and management to ensure that KPI results are understood and translated into meaningful actions. Quality assurance does not operate in isolation but rather provides data and analyses so that you can make well-founded decisions. This way, the KPI system remains a means of supporting corporate management and does not become a bureaucratic exercise.
Basic principles for setting up a KPI system
A few basic principles should be kept in mind when setting up or further developing a KPI system for your company. These guidelines will help ensure your KPI system is targeted and effective:
- Clear target orientation: Every KPI should clearly reference specific company or process targets. KPIs only have strategic relevance if they serve as a control instrument for achieving objectives. Therefore, ask yourself the following question for every KPI: Which objective is being monitored? Without this reference, the value remains a descriptive figure with no operational benefit. KPIs should be derived directly from your most important success factors and focus on what truly matters for your company’s success.
- High data quality: Ensure high data quality, as this is the basis of your KPIs. Every metric must be clearly defined and reproducibly collected. “Garbage in, garbage out”: Poor or incomplete data inevitably leads to misinterpretations and incorrect conclusions. Therefore, ensure that the data sources are well-maintained and that measurement methods are standardized. Only valid and accurate information will provide you with a reliable picture. If in doubt, do not hesitate to invest in improving data collection before making far-reaching decisions based on figures.
- Clear responsibilities: Determine who is responsible for collecting, monitoring and evaluating each KPI. KPIs are only effective if it is clear who will initiate measures in the event of negative trends. Therefore, appoint someone responsible for each KPI who will monitor developments and react in the event of deviations from the target. This personal accountability also promotes acceptance of the KPIs within the team because those involved know what they are being measured against and can contribute proactively. Additionally, management should regularly provide feedback on the KPIs in order to emphasize their importance.
- Appropriate number and structure: Less is more. Concentrate on a few high-quality KPIs rather than many low-quality ones. An overly complex KPI system harbors the risk of “analysis paralysis”: the flood of data paralyzes decisions instead of promoting them. It is better to define a manageable number of really meaningful KPIs that examines issues from different perspectives (e.g. quality, efficiency, customer satisfaction and profitability). Ideally, KPIs should be coordinated within a KPI system to provide a holistic picture and prevent the creation of isolated silos of figures. Methods such as the balanced scorecard help you to make balanced KPI selections that combine different target areas of the company. It is important that you derive actions from each KPI – otherwise they are unnecessary.
- Regular review and adjustment: A KPI system is not static, so regular review and adjustment are important. Therefore, you should regularly check whether your KPIs are providing the desired insights. Question the KPI selection during management reviews. If certain KPIs are not discussed for an extended period or do not provide added value, do not hesitate to revise or replace them with more effective indicators. In dynamic industries or in the case of company growth, KPI systems must adapt to changing targets and grow with the company. Note that hard KPIs often need soft factors to supplement them because not everything important can be expressed as a number. The key is to continuously learn from and develop the KPI set so that it best supports your strategic initiatives.
Quality criteria for effective KPIs
Not every metric is automatically a good KPI. In order to be a suitable KPI, a metric must fulfill certain quality criteria. The most important criteria for an effective KPI are listed below:
- Relevance: The KPI must be significant for the company or process. It should measure an aspect that significantly contributes to success or the achievement of objectives. A KPI is only effective if it directly targets a priority, such as improving performance, meeting quality standards, ensuring patient safety, or achieving regulatory compliance. Without this reference, the KPI is interesting, but not a management tool. Therefore, determine whether an indicator is important enough to report regularly to management.
- Comparability: KPIs must be defined consistently so that comparisons can be made across time periods, departments, and benchmarks. Uniform definitions and standardization are essential for this. Only comparable KPIs allow you to identify trends or derive best practices. For example, use a reference value (percentage or rate per X units) instead of an absolute figure without context. Instead of just looking at the number of defects, for example, you could define the defect rate per 1,000 production units, which makes the metric comparable and interpretable across different production volumes.
- Influenceability: A KPI should be influenced by the actions of those responsible. KPIs whose value cannot be influenced by the team are not suitable for management purposes because they only describe a status and offer no opportunity for improvement. Therefore, ask yourself: “Can we change this value through measures?” If not, then the figure belongs in monitoring for informational purposes, but not in the management cockpit. For example: The number of external audits that an authority carries out each year is outside your sphere of influence and would not be a suitable KPI for your company. However, the error rate in production can be reduced through training, process optimization, or better machine maintenance. Therefore, it can be influenced and is a suitable KPI, provided it is relevant.
- Validity: Validity refers to whether the measurement accurately reflects the desired aspect. The selected metric should allow you to draw a direct conclusion about the desired aspect. Critically question whether the KPI captures the intended construct or if distorting factors play a role. Example: A high number of completed training courses per employee may indicate employee qualifications. But is this KPI valid? Completed training courses are only valid indicators of competence if the training content is relevant and effective. Without content quality assurance, the “number of training courses” KPI measures diligence rather than understanding. As you can see: A good KPI requires a clear definition so that the figures can actually be used to make statements about the desired success-critical factor later.
- Timeliness: KPIs should be measured promptly and at reasonable intervals so that you can react quickly to deviations. Outdated data loses its control value. Therefore, you should define the intervals at which the KPI is updated (e.g., monthly or quarterly) and ensure that it is reported to decision-makers in a timely manner. Only up-to-date KPIs allow you to identify trends early on, before opportunities are missed or risks materialize.
- Comprehensibility and acceptance: Create additional transparency about your KPIs to promote acceptance. Everyone involved, from workers to management, should understand what is being measured and why it is relevant. A KPI that nobody understands or rejects within the company will hardly achieve the desired control effect. Document definitions and objectives and train your team to use KPIs. If everyone recognizes the importance of KPIs and their benefits, the results will be seen as motivation for improvement rather than as an annoying form of control.
Strategic vs. operational KPI levels in quality management
KPIs can be used at the strategic and operational levels. Strategic KPIs measure the achievement of an organization’s long-term, overarching goals, while operational KPIs measure the performance of specific processes and short-term goals. In the context of ISO 13485, strategic KPIs often relate to overarching quality objectives, such as reducing the complaint rate or ensuring compliance with regulatory requirements. Operational KPIs, on the other hand, measure the effectiveness of quality processes in day-to-day business, such as the average CAPA processing time
Strategic KPIs: They are derived from the quality policy and the company’s objectives. Upper management uses them to control overall quality success. Examples:
- Audit success rate (percentage of audits without serious deviations – an indicator of robust compliance),
- Time-to-market for new products (the time it takes to receive market approval) shows the efficiency of the development and approval process,
- Customer complaint rate (number of complaints per unit sold), which reflects customer satisfaction and product quality.
These key figures show whether the QM system as a whole is delivering the desired results.
Operational KPIs: These KPIs monitor specific processes and workflows. They are relevant for process managers and help to identify deviations in day-to-day operations early on. Examples:
- Throughput time of a CAPA process (shows the efficiency of troubleshooting),
- Number of open deviations in a process (indicates whether problems are being addressed promptly),
- Percentage of production plant maintenance carried out on schedule (monitors plant availability).
Such KPIs enable process-oriented intervention and continuous improvement in the operational area.
Evaluation of specific KPIs: suitability and pitfalls
To help you better understand the principles and criteria outlined above, we will conclude by examining some specific KPIs from the MedTech sector. We will explain how suitable each of these KPIs is for process evaluation and where potential misinterpretations lurk. Always remember: KPIs need context. A seemingly good or bad value does not tell the whole story if the circumstances are not considered.
- Complaint Rate: This metric indicates the number of customer complaints relative to products delivered. It is an important indicator of product quality and customer satisfaction, making it a relevant KPI in the field of medical devices.
Suitability: A falling complaint rate may indicate improved product quality, while a comparatively high value may signal the need for action in areas such as development, production, or support.
Possible misinterpretation: The complaint rate should always be considered relative to the intensity of use. For instance, if your company is experiencing strong growth and more products are in circulation, the absolute number of complaints may increase, even if the rate per device remains stable. Conversely, a very low complaint rate could mean that users are not reporting problems (e.g., because the reporting process is too time-consuming). It would be misleading to evaluate employees based on how few complaints they receive because this could lead to problems being covered up or customers being deterred from complaining. Therefore, use this metric with caution, and pay attention to supplementary feedback, such as that from customer satisfaction surveys.
- Internal Audit Findings: Internal audits reveal deviations and weaknesses in quality management. One key performance indicator could be the number of audit findings per audit or per year.
Suitability: Audit findings demonstrate how well your QMS meets the requirements. A decrease in the number of serious deviations over time can indicate effective improvement work.
Possible misinterpretation: Fewer reported deviations do not automatically indicate better quality; they could also mean that less attention was paid. If your company’s prevailing culture is to “find as few errors as possible” to appear competent, then there is untapped potential for improvement. Conversely, an increase in identified deviations should not be prematurely interpreted as a deterioration. It depends on the context. Were new areas examined in the audit? Was the bar raised? Has the audit team been trained to conduct more thorough audits? In fact, an increase in detected deviations during the initial period can be a good sign that your monitoring system is effective and increasing transparency. For example: In one company, the reporting procedure for quality incidents was simplified, and employees were made more aware of the issue. The following year, the number of reported incidents increased significantly. However, this KPI value, which initially appeared negative, actually indicated an improved reporting culture and revealed previously hidden problems. This example shows that a deterioration in individual KPI values does not necessarily mean a step backward, as long as the context is understood. It also underscores the importance of always interpreting KPIs in the overall context.
- CAPA Lead Time: CAPA stands for Corrective and Preventive Action. These are the corrective and preventive measures derived from error messages or audits. A common KPI is the average time for CAPA processing, from problem detection to completion of the measure.
Suitability: Processing time reflects the responsiveness of your quality management. Shorter processing times indicate that problems are analyzed and resolved quickly. This is extremely important in a regulated industry.
Possible misinterpretation: However, focusing solely on quickly completing CAPAs can create the wrong incentives. If employees only focus on meeting deadlines, they may “complete” measures prematurely without eliminating the underlying causes, allowing quality problems to occur again later. Quality problems may then occur again later. Therefore, evaluate the effectiveness of the measures in addition to the speed (e.g., verify the effectiveness in the CAPA process). Another important point is that a longer CAPA process isn’t necessarily a bad thing. Complex, systemic problems sometimes require more time to find a sustainable solution. It is important to consider the appropriate timeframe and, if necessary, set interim targets instead of fixing them on a monthly value.
- Defect Rate in Production (Reject Rate): This KPI measures the proportion of devective products or components in the production process.
Suitability: The error or reject rate is a key quality indicator in production. A low rate indicates efficient processes and high product quality, as well as fewer reworks and rejects.
Possible misinterpretation: It is important to understand how the ratio is calculated. Is the KPI recorded stringently and consistently? Are there clear criteria for what constitutes an error? A common pitfall is employees only “working towards” the KPI, such as by not reporting faulty parts to keep the rate low. This distortion can occur if reporting errors is viewed as a sanction rather than an opportunity in the corporate culture. Therefore, create an environment in which accurate data is valued over polished numbers. Only then will the error rate provide reliable information. The economic optimum rate should also be considered. While a reject rate of 0% may be ideal, the effort and cost to achieve it could be disproportionate. Setting target values that consider both quality and cost-effectiveness is helpful.
Management review - requirements and meaningful KPI reports
In accordance with ISO 13485, the management review is top management’s formalized evaluation of the QMS and its performance. All KPIs converge here. The standard stipulates that management must regularly evaluate a wide range of input data to ensure the QMS’s suitability, appropriateness, and effectiveness. The twelve prescribed inputs of a management review include, in particular:
- Feedback, complaints and reports: including all customer feedback data, complaints, and trends from post-market surveillance, as well as official reports, if applicable. These show how the product is performing in the field and where there are risks or room for improvement.
- Audit results: Findings from internal and external audits of the QMS. The focus is on the number and severity of audit findings and the corrective measures implemented.
- Monitoring the processes: Key performance indicators of the most important QMS processes (KPIs) should be included. Management should see at least one KPI for each core process to assess whether the processes met the requirements during the previous period. For example, the following is reported: CAPA average time vs. target, number of open CAPAs, complaint rate, production rejection rate, supplier quality, etc. These KPI trends provide an objective picture of process performance.
- Status of corrective and preventive actions: This provides a higher-level view of the effectiveness of the CAPA system. For example, have there been trends in nonconformities? Have CAPAs from the previous period been successfully completed, and have they had the desired effect? Management is particularly interested in whether recurring problems are under control and if there are any systematic weaknesses.
- Follow-up actions from previous reviews: Measures decided upon at the last management review must be tracked. One key performance indicator (KPI) for this is the completion rate of the previous year’s action items (%). Ideally, it would be 100%, or there would be a clear explanation if something was postponed.
- Changes that could affect the QMS: e.g. new or amended regulatory requirements, organizational changes, new products/processes that need to be taken into account. Although these are not KPIs in the strict sense, they must be reported qualitatively.
- Recommendations for improvement: All internal inputs, such as employee suggestions or CIP, or external best practices that could improve the QMS, should be incorporated. Key figures on submissions and implementation provide a quick overview here, too.
These inputs are discussed and evaluated by management during the management review. The agreed-upon output actions resulting from this review must be documented. ISO 13485 stipulates that the following results of the review must be recorded at a minimum:
- Measures to improve the QM system and its processes – e.g. initiatives to make processes more efficient or more compliant, based on KPI trends and supported by metrics that measure progress.
- Measures to improve products in terms of safety and performance – such as product modifications or additional checks, if complaint analyses have revealed patterns, for example.
- Need to change resources – if the key figures show that certain areas are under- or overloaded or if new skills or tools are required (e.g. more staff for CAPA processing or the purchase of a software tool for data analysis).
- Possible adjustments to quality targets or policy – if previous KPIs show that targets were set too low or too high, management can set new targets or strategies. Trends in KPIs over several years are particularly informative for this purpose, showing whether quality is improving, stagnating, or deteriorating
A well-prepared management review clearly presents KPIs using graphs, trends, and traffic light indicators. It focuses on outliers and negative trends, as well as the decisions derived from them. Ultimately, the review ensures that the data cycle is closed. KPI-based findings lead to management decisions, which lead to improvement measures in line with the ISO 13485 continuous improvement process (PDCA cycle).
Practical tip: Prepare a key performance indicator scorecard or dashboard for the management review that compares the most important KPIs from the previous period with target values. This allows management to recognize trends at a glance. For example, you can display your data using traffic light colors: Green = target achieved, yellow = within tolerance, and red = action required.) This makes it easier to focus on the red areas during the meeting
Role of digital tools and software in KPI collection and evaluation
In the digital age, the collection and evaluation of strategic and operational KPIs can be significantly simplified and improved. According to a Deloitte study, ~33% of companies state that their controlling processes are associated with a very high level of manual effort, and many companies still struggle with manual processes when it comes to data collection and preparation of key figures. Modern digital solutions offer significant advantages for navigating the “data jungle.”:
- Automated data acquisition: Specialized QMS software (e.g., Orcanos, MasterControl, and Greenlight Guru) extracts and logs quality data directly from process data to visualize it in real time and make it available for further analysis. Rather than laboriously compiling KPI data in Excel spreadsheets, the data is available in real time in respective modules (e.g., Complaints Management, CAPA Control, Design Control, etc.) in a central database. This reduces errors and effort while ensuring you always have up-to-date figures.
- Free KPIs: Many KPIs are included in digital management systems without requiring separate planning. For example, the sheer number of data objects, such as CAPAs, complaints, or defects, allows you to draw conclusions about quality KPIs. Together with points in time, such as start and end dates and due dates, as well as the process steps a data object must go through, a wide range of statements can be made about process quality. Once again, the more granular the recorded data and information, the greater the possibilities for meaningful analyses.
- Dashboards and visualization: Modern BI tools (Power BI, Tableau, etc.) and the integrated dashboard functions of QMS platforms clearly display KPIs. Interactive dashboards offer drill-down options that allow users to navigate from an overview KPI (e.g., the total number of open CAPAs) to the details (e.g., which CAPAs are overdue and in which department most occur). This gives teams at all levels the depth of information they need.
- Real-time alerting and notifications: Digital tools can send and escalate alerts when certain KPI thresholds are exceeded (e.g., “CAPA older than 30 days” or “defect rate greater than x% this week”). This enables a proactive response before a problem escalates.
- Standardized reports at the touch of a button: Rather than manually creating the management review report, many QMS systems can generate predefined reports containing all the necessary KPI data in accordance with ISO 13485 specifications at the click of a button. This saves time and ensures nothing is overlooked. At the same time, these tools provide audit trails and evidence proving to auditors that the reviews have been completed.
- Integration and data merging: KPI software can often be linked to other systems, such as ERP, LIMS, and production systems. This allows quality KPIs to be analyzed alongside production or financial data, such as quality costs, rejects versus production batches, and supplier performance versus delivery delays. This holistic view recognizes correlations that isolated observations would not reveal.
- Trend analysis and forecasting: Modern tools offer analytical functions such as trend curves, statistical evaluations, and in some cases, AI-supported analyses. These functions allow you to recognize patterns (e.g., seasonal effects or creeping deterioration) and make predictions, such as forecasting when a process target would be missed without countermeasures. Such data-driven insights enable preventive action instead of reactive action.
In summary, digital tools support more efficient KPI management. They reduce the effort required to collect data, increase its accuracy, and make key figures more accessible through visualization. For managers, this means spending less time collecting data and more time analyzing it and making decisions. This is how Deming’s opening quote is put into practice: data becomes insights, and insights become improvement actions.
Conclusion
KPI-driven company management combines compliance with business excellence. It fulfills the ISO 13485 standard’s obligation to monitor processes and delivers tangible benefits to the company. Strategic KPIs link quality objectives with corporate goals, and operational KPIs provide insight into process performance.
When used correctly, KPIs provide clear insights into your processes’ performance and compliance and help you make informed decisions. However, remember that KPIs are only a means to an end. Your ultimate goal is to transform data into decisions, drive improvements, and maintain your organization’s course. The best way to ensure sustainable success is to focus on a few meaningful KPIs combined with common sense and an open culture of improvement. If KPIs are viewed in isolation, however, they can provide a distorted picture or create false incentives. Prevent this by always interpreting your KPIs in context, questioning their quality, and viewing them as an integral part of a holistic management system.
Although the introduction of good KPIs initially requires effort, with the right selection (SMART and relevant) and modern tools, they become a management instrument that can provide competitive advantages, such as greater product safety, more efficient processes, and more satisfied customers.
Ultimately, you can’t improve what you don’t measure. KPIs provide the measuring points and create the basis for systematically developing quality management further. Use KPIs to turn data into concrete action.
