- Journal List
- HHS Author Manuscripts
- PMC8689590
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
AAPS J. Author manuscript; available in PMC 2022 Nov 12.
Published in final edited form as:
AAPS J. 2021 Nov 12; 23(6): 117.
Published online 2021 Nov 12. doi:10.1208/s12248-021-00657-y
PMCID: PMC8689590
NIHMSID: NIHMS1761423
PMID: 34773177
Sean A. VanDuyse, Michael J. Fulford, and Michael G. Bartlett
Author information Copyright and License information PMC Disclaimer
The publisher's final edited version of this article is available at AAPS J
Associated Data
- Supplementary Materials
Abstract
Purpose:
The International Council for Harmonization (ICH) “Q10 Pharmaceutical Quality Systems” (ICH Q10) guidance was introduced to address the growing gap between current good manufacturing practices and pharmaceutical manufacturing quality systems. This study evaluated the impact of the ICH Q10 guidance on the PQS of pharmaceutical manufacturers.
Materials and Methods:
Data were obtained from the enabler questionnaire from pharmaceutical manufacturers surveyed by the St. Gallen OPEX Benchmarking Program. These results represent the degree of implementation for enabler-focused questions based on a 5-point Likert scale self-assessment. Data analysis included a comparison of means and medians before and after the release of the ICH Q10 guidance and annual changes.
Results and Discussion:
There was a statistically significant difference for enabler implementation as a whole (p-value < 0.0000), before and after the release of ICH Q10 Furthermore, statistically significant differences were observed for four of the five enabler categories (p-values <0.05). In particular, the EMS enabler category showed a decrease in mean enabler score, suggesting the Management Responsibilities ICH Q10 PQS element was not effectively described or implemented.
Conclusion:
These results indicate that the release of ICH Q10 had a positive impact on the PQSs of pharmaceutical manufacturers. This was driven primarily by the changes observed in the TQM and JIT enabler categories and complimented by the TPM and BE categories. This would suggest that the Management Review, Change Management System, and Process Performance and Product Quality Monitoring System ICH Q10 PQS elements were all effectively described and implemented.
INTRODUCTION
There have been recent advancements in manufacturing science and an increasing understanding of the positive impact of quality systems across the pharmaceutical industry. Accompanying this is a need for guidance from regulatory authorities on the appropriate implementation and maintenance practices for Pharmaceutical Quality Systems (PQS). This need is currently being addressed by individual regulatory body projects, such as the United States Food and Drug Administration’s (FDA) Pharmaceutical cGMPs for the 21st Century Initiative [(1)]. Additionally, international harmonization efforts have been made towards this need, as seen by the International Council on Harmonization (ICH) and International Organization for Standardization (ISO) guidance documents related to PQS.
Despite the numerous initiatives and guidance, a review of the overall trends in FDA 483 observations and warning letters reveals that cGMP issues, particularly inadequacies that should be addressed by a complete and effective PQS, continue to be the most frequent infractions. Examples of the most frequent observations include “quality control procedures not in writing or not fully followed” for drug products and “lack of or inadequate CAPA procedures” for device products [(2)]. Therefore, more work must be done by both the pharmaceutical industry and the regulatory agencies to mitigate these continuing violations. Insight into the effectiveness of past and current projects will allow for the use of the more effective methods and a reevaluation of the less effective methods.
The purpose of this research is to determine if the ICH “Q10 Pharmaceutical Quality Systems” guidance document had a statistically significant impact on the PQS of manufacturing sites around the world. This was determined by the evaluation of the degree of difference in production principles and observable behaviors between manufacturing sites prior to ICH Q10 publication and sites after ICH Q10 publication using data from the St. Gallen OPEX Data Benchmarking Questionnaire database. This study determines the statistical significance and degree of difference in the means of the responses before ICH Q10 and after ICH Q10 release. Furthermore, this study evaluates the statistical significance and degree of difference in the means of each of the five enabler categories, as defined by the St. Gallen OPEX group. Overall, the identification and analysis of the impact of ICH Q10 provides insight into the effectiveness of this guidance document in improving pharmaceutical manufacturing site quality systems to produce more safe and effective products.
BACKGROUND
2.1. Quality in the Pharmaceutical Industry
Due to the nature of pharmaceutical products, it is of utmost importance to maintain quality in all aspects of their manufacture. The quality of these products can be evaluated through the combination of the products established identity, strength, purity, and other quality characteristics, which are designed to ensure the required levels of safety and effectiveness [(3)]. The approach to achieve this is a system of programs, policies, processes, and facilities that ensure quality. This system is known today as the Pharmaceutical Quality System (PQS). This idea is based on the premise that quality should be built into the product, as in many cases testing alone cannot be relied on to ensure product quality [(4)]. Effectively and efficiently implementing PQS principles is not a simple task. Therefore, both national regulatory bodies and international groups have done tremendous work on developing regulations and guidance to facilitate the quality systems of pharmaceutical manufacturers [(3, 5, 6)].
2.2. Description of ICH Q10 Pharmaceutical Quality Systems
The U.S. Food and Drug Administration (FDA) and other national organizations are acutely aware of the importance of pharmaceutical quality and have taken measures to ensure and promote quality. One of these measures is the International Conference on Harmonization of technical requirements for registration of pharmaceuticals for human use (ICH). This project assembles the regulatory authorities of Europe, Japan and the United States as well as experts from the pharmaceutical industry of these three different regions. These groups then collaborate for discussions of the scientific and technical aspects of pharmaceutical product registration. An objective of the ICH is to develop guidelines on the quality, safety, and efficacy of pharmaceutical products. These guidelines are subsequently adopted by ICH regulatory members and observers resulting in international harmonization.
The ICH “Q10 Pharmaceutical Quality Systems” is an essential guidance document for pharmaceutical manufacturers attempting to develop systems to ensure the quality of their products. This document was finalized in June of 2008 and was later implemented by the European Commission in July of 2008 and the FDA in April of 2009. A full visualization of the implementation history of ICH Q10 is presented in figure 1 [(6)].
Figure 1.
Timeline of implementation of ICH Q10 by major regulatory agencies.
The intended purpose of ICH Q10 is to assist pharmaceutical manufacturers in designing and implementing an effective quality management system. ICH Q10 attempts to fulfil this purpose by detailing a model pharmaceutical quality system (PQS), also referred to as the ICH Q10 model. This model is centered on International Organization for Standardization (ISO) quality concepts and can be implemented during the product lifecycle’s different stages. An effective PQS is described in ICH Q10 Section 3.1.3 as a system that “assures that the desired product quality is routinely met, suitable process performance is achieved, the set of controls are appropriate, improvement opportunities are identified and evaluated, and the body of knowledge is continually expanded.” [(6)].
The elements of an effective PQS, as described in the ICH Q10 guidance are as follows: Management Responsibilities, CAPA system, Process Performance and Product Quality Monitoring System, Change Management System, and Management Review [(6)]. In addition to these elements, ICH Q10 emphasizes the importance of and interconnectedness of Knowledge Management and Quality Risk Management to a successful and effective PQS, which are both further described in the sister guidance documents ICH “Q8 Pharmaceutical Development” and ICH Q9 “Quality Risk Management”, respectively [(4)]. These diverse components of the ICH Q10 model are the result of its basis in system theory. The system approach emphasizes a holistic evaluation of the complex interrelationships and various influences that compose a system. Each component is not working in a vacuum but rather supporting others and in turn being supported.
Management Responsibilities
The ICH Q10 model provides details on management responsibilities that are critical to the performance of the PQS. Typical management responsibilities are also described in this guidance. These responsibilities include resource management, internal communication, and management review. Resource management is defined as the determination of what resources are needed for a given process or activity and the subsequent provision of these resources [(6)]. It also is important to ensure that the resources are being utilized appropriately and effectively. Internal communication involves the establishment of appropriate communication processes. These communication processes allow for the flow of appropriate information between levels and units of the organization. They are also the pathway for product quality or PQS issue escalation. Management review is a responsibility of senior management, which allows for continued improvement and sustained suitability of both the manufacturing processes and the PQS [(6)]. It is also identified as one of the four enhanced PQS elements of the ICH Q10 model and will be discussed further in the evaluation of these PQS elements.
Process Performance and Product Quality Monitoring System.
The Process Performance and Product Quality Monitoring System elements allows the PQS to maintain a state of control. In order to achieve this, the monitoring system must both provide assurance of the continued capability of the process and identify areas for continual improvement. One of the responsibilities of this system is developing the data management and statistical tools for measurement and analysis of parameters and attributes identified in the control strategy. Another responsibility is the gathering of feedback on product quality from both internal and external sources for enhancement of process understanding.
Change Management System
An effective change management system enables the organization to evaluate, approve, and implement changes appropriately. One of the key activities of this system are the use of subject matter experts and diverse teams to contribute to the evaluation of the proposed changes. Another is the monitoring and evaluation of the change after it is implemented, which allows for the determination of whether change objectives were achieved and if there were any harmful impacts on product quality [(6)]. These components align to allow change management systems to implement new directions that facilitate continual improvement and assure that there are no unintended consequences of these changes.
Management Review
The last PQS element enhanced by the ICH Q10 model is management review of process performance and product quality. The review should include the results of regulatory inspections, audits, and periodic quality reviews. The periodic quality reviews allow for evaluation of the other systems, such as the effectiveness of process and product changes originating from the CAPA system and the findings of the process performance and product quality monitoring system. Some of the actions that management review is responsible for are improving the manufacturing processes and the reallocation of resources to better fit the process [(6)]. Through these activities, this system works as the head of the PQS to provide assurance that process performance and product quality are managed over the lifecycle of the product.
2.3. Description of the St. Gallen OPEX Benchmarking Program
The St. Gallen OPEX Benchmarking Program was initiated in 2004 as an international research project to examine the link between the performance scores and the enablers leading to this performance. Their research program involves the collection of both Key Performance Indicators (KPIs) and responses to the enabler questionnaire. These data were then used to develop the Pharmaceutical Production System Model (PPSM). The PPSM provides a structured and holistic depiction of the relevant, available data from the St. Gallen OPEX Database. This allows for the structured analysis of the components, which support the specific achievement of PQS Excellence. This research analyzed one of these components, the questionnaire data from the qualitative enablers within the Cultural Excellence category of the PPSM. We acknowledge the limitations of analyzing a single component rather than the complete model and conclusions drawn from this analysis have taken these limitations into account.
The questionnaire data were composed of the responses to a number of questions regarding enablers from pharmaceutical manufacturing sites around the world. Enablers are defined by St. Gallen as “production principles (methods & tools but also observable behavior). The values show the degree of implementation based on a self-assessment on a 5-point Likert scale.” [(7)]. These enablers are divided into the following 5 categories: Total Productive Maintenance (TPM), Total Quality Management (TQM), Just-in-Time (JIT), Effective Management System (EMS), and Basic Elements (BE). The CAPA systems of the manufacturing sites captured through the St. Gallen OPEX benchmarking study are evaluated using KPIs such as the number of CAPAs as well as a calculated “Supplier Reliability Score” [(7)]. As such, these values were not compatible with the analysis performed on the questionnaire data and were excluded from this study. Additionally, while Knowledge Management and Quality Risk Management are fundamental to the implementation and interpretation of ICH Q10, no conclusion was drawn to the effectiveness of these guidance to maintain a clear and concise scope for the study. It is important to clarify that the St. Gallen OPEX enablers and the ICH Q10 PQS enablers are distinct. This research focused on evaluating the impact of the ICH Q10 release on the PQS elements of pharmaceutical manufacturing sites through the analysis of St. Gallen enabler implementation. We acknowledge that the tight scope of the analysis limits conclusions made on the impact guidelines as it may not be the true and exclusive cause of observed changes. Further research into evaluating the impact of the other Quality Guidelines, particularly ICH Q8 and Q9, is needed to understand the complex influences being had on PQS development and maintenance.
Total Productive Maintenance
TPM is a comprehensive approach to equipment ready maintenance that emphasizes proactive and preventative maintenance, in order to maximize the operational efficiency of equipment. The TPM enabler category is divided into three sub-categories in the St. Gallen Enabler Questionnaire: Preventative Maintenance, Housekeeping, and Effective Technology Usage. Each of these sub-categories is designed to evaluate methods that ensure a high level of equipment stability and availability [(6)]. This Enabler Category does not have a direct comparison to any specific ICH Q10 PQS Elements as it is a manufacturing philosophy.
Total Quality Management
The overarching objective of TQM is to reduce process variability and thereby increase process stability [(8)]. Some of the TQM techniques to improve operational performance include scientific methods for work organization, monitoring, and value analysis of work processes. The TQM enabler category is designed to assess the necessary practices to stabilize the manufacturing equipment or ensure robust supporting processes [(7)]. This enabler category is composed of the following four sub-categories: Process Management, Customer Integration, Cross-Functional Product Development, and Supplier Quality Management.
Just-In-Time
The objective of JIT is to establish an advantage through the delivery of superior products or services in terms of both cost and quality [(9)]. This objective can be achieved through the pursuit of several specific goals, those being the continual elimination of waste, improvement of product quality, and maximization of production efficiency. However, there are some limitations to JIT which include certain prerequisites to implementation, increased dependence on the consistency of supply chains, and the loss of the buffer against supply/demand fluctuations associated with safety stocks or excess capacity [(6)]. The OPEX benchmarking group describes the prerequisites to JIT implementation in the following quote “Only after both equipment and processes are stabilized, can Just-In-Time (JIT) production potentially be achieved within a production environment.” [(10)]. This is because in order to reduce the unnecessary work-in-progress inventory and minimize cycle times, JIT is reliant on robust processes. Without robust processes, the increased stress placed on the production system and lack of safety stocks to compensate for operational instabilities can result in product delivery disruptions and even shortages of important pharmaceutical products [(10)]. The JIT enabler category is broken into four sub-categories. These sub-categories are Set-Up Time Reduction, Pull Production, Layout Optimization, and Planning Adherence.
Effective Management System
The EMS enabler category was designed to assess the management systems and capabilities of the manufacturing sites. The objective of EMS is described by Friedli and co-editors in their book Leading Pharmaceutical Operational Excellence as “to motivate and align employees to work for a common goal” [(11)]. The EMS enabler category contains four sub-categories. These being Direction Setting, Management Commitment and Company Culture, Employee Involvement and Continuous Improvement, and lastly, Functional Integration and Qualification.
Basic Elements
The BE enabler category is defined by the St. Gallen OPEX group as representing “a collection of practices that are shared by all three technical categories (TPM, TQM and JIT), including the implementation of fundamental OPEX practices such as Standardization and Visual Management” [(10)]. The BE enabler category is divided into the following two sub-categories: Visual Management and Standardization & Simplification. A comparison of the ICH Q10 PQS elements and the associated St. Galen Enabler group is presented in Figure 2.
Figure 2.
ICH Q10 PQS elements and St. Gallen Operational Excellence (OPEX) benchmarking enabler categories comparison
3. METHODS, RESULTS, AND DISCUSSION
3.1. Methods
This article does not contain any studies with human or animal subjects performed by any of the authors.
This study involved the analysis of the results from 358 responses to the St. Galen OPEX benchmarking questionnaire, related to more than 330 pharmaceutical manufacturing sites (2003 – 2018) [(10)]. These data were obtained through collaboration with the Operational Excellence team of the Institute of Technology Management at the University of St. Gallen (ITEM-HSG). The database consisted of 5-point Likert scale responses to the survey questions regarding enablers. Total responses for all enablers before 2009 were combined into one data set and the same was done for the responses from 2009 and later. These data were then transferred from spreadsheets into the Stata statistics software to conduct a series of statistical tests including histograms to visualize the spread, descriptive and summary statistics, and comparing means and medians. Further analysis was performed using these same techniques on subsets of the samples to analyze if significant differences can be observed at the enabler category level. Additionally, the mean, median and standard deviation were determined for the responses from each year. This was performed to determine if there was an overall trend of improvement over time for the PQSs represented in the sample, which could be an alternative explanation for any observed statistically significant difference between the Pre-09 and Post-09 groups.
The statistical tests performed include a two-sample t-test assuming unequal variances, a two-sample z-test and the Wilcoxon signed rank test. The z-test and t-test were performed to determine whether the means for enabler response of these two groups, before 2009 and after 2009, are equal. Therefore, the tests were performed as two-tailed tests. The results of these tests allowed us to identify statistically significant differences between the means of the group, and the group with the higher value. The Wilcoxon signed rank test was performed to provide an alternative for the t-test when the distribution of the differences between the two samples cannot be assumed to be normally distributed. This test is a comparison of medians between the samples and would allow us to confirm if there is a statistically significant change in measures of central tendency from the pre-2009 group to the post-09 group. For all tests, a significance level of α = 0.05 was used. These comparisons were also performed on subsets of the total group to analyze if the measured change observed at the combined level occurs at the enabler category level.
3.2. Complete Population Results
The summary statistics of the complete population analysis groups are presented in Table 1.
Table 1
Complete Population Summary Statistics
| GROUP | # OF OBSERVATIONS | MEAN | MEDIAN | STANDARD DEVIATION |
|---|---|---|---|---|
| BEFORE 2009 | 16,840 | 3.297 | 3 | 1.268 |
| AFTER 2009 | 19,492 | 3.409 | 4 | 1.264 |
Open in a separate window
The after 2009 group had a sample mean of all question responses 0.112873 greater than the before 2009 sample group. This mean difference is supported by the results of the statistical tests, presented in Table 2.
Table 2
Complete Population Statistical Test Results
| GROUP | P-VALUE OF T-TEST | P-VALUE OF Z-TEST | P-VALUE OF WILCOXON TEST | T SCORE OF T TEST | Z SCORE OF Z TEST | Z SCORE OF WILCOXON TEST |
|---|---|---|---|---|---|---|
| BEFORE 2009 VS AFTER 2009 | < 0.0000 | < 0.0000 | < 0.0000 | 8.4754, d.f.: 35532.7 | 10.729 | 8.987 |
Open in a separate window
*Df are Satterwhite’s degrees of freedom calculated by Stata.
Based on these results we can reject the null hypotheses of the t and z tests that the means of the two groups are equal and conclude that the observed 0.113 difference is statistically significant. Furthermore, we can reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that the observed difference in median is statistically significant.
Figure 3 presents histograms of all responses from both the before 2009 and after 2009 groups for the purpose of visualizing the observed changes. This figure shows the increased proportion of responses scored as five and the decreased proportion of responses scored as one or two. This suggests that observed differences are a result of more complete enabler implementation.
Figure 3.
Pre 2009 and Post 2009 complete population histogram comparison
3.3. Enabler Category Results
The summary statistics of the enabler category analysis groups are presented in table 3.
Table 3
Enabler Category Summary Statistics
| GROUP | # OF OBSERVATIONS | MEAN | MEDIAN | STANDARD DEVIATION |
|---|---|---|---|---|
| BEFORE 2009 TQM | 3,827 | 3.299 | 4 | 1.280 |
| After 2009 TQM | 4,033 | 3.591 | 4 | 1.298 |
| BEFORE 2009 TPM | 2,971 | 3.333 | 4 | 1.305 |
| AFTER 2009 TPM | 2,662 | 3.359 | 4 | 1.242 |
| BEFORE 2009 JIT | 3,735 | 3.004 | 3 | 1.264 |
| After 2009 JIT | 4,214 | 3.086 | 3 | 1.277 |
| BEFORE 2009 BE | 2,288 | 3.315 | 4 | 1.254 |
| AFTER 2009 BE | 3,578 | 3.495 | 4 | 1.240 |
| BEFORE 2009 EMS | 3,590 | 3.663 | 4 | 1.099 |
| AFTER 2009 EMS | 4,548 | 3.651 | 4 | 1.095 |
Open in a separate window
The Total Quality Management (TQM) enabler category had a 0.292 difference in mean. The Total Productive Maintenance (TPM) enabler category had a 0.026 difference in mean. The Just-In-Time (JIT) enabler category had a 0.0826 difference in mean. The Basic Elements (BE) enabler category had a 0.181 difference in mean. Lastly, the Effective Management System (EMS) enabler category had a 0.013 difference in mean. No enabler category had a change in median. The results of the statistical analysis are presented in Table 4.
Table 4
Enabler Category Statistical Test Results
| GROUP | P-VALUE OF T-test | P-VALUE OF Z-TEST | P-VALUE OF WILCOXON TEST | T SCORE OF T TEST | Z SCORE OF Z TEST | Z SCORE OF WILCOXON TEST |
|---|---|---|---|---|---|---|
| TQM | < 0.0000 | < 0.0000 | < 0.0000 | 10.0418, d.f. 7846.42 | 12.937 | 9.126 |
| TPM | 0.4455 | 0.3317 | 0.0003 | 0.7631, d.f. 5610.25 | 0.971 | 3.651 |
| JIT | 0.0039 | 0.0003 | 0.0408 | 2.8834, d.f. 7850.35 | 3.662 | 2.046 |
| BE | < 0.0000 | < 0.0000 | 0.2719 | 5.4020, d.f. 4832.61 | 6.745 | 1.099 |
| EMS | 0.6066 | 0.572 | 0.5803 | 0.5150, d.f. 7692.72 | 0.565 | 0.553 |
Open in a separate window
*Df are Satterwhite’s degrees of freedom calculated by Stata.
Total Quality Management
Based on these results we can reject the null hypotheses of the t and z tests that the means of the two TQM analysis groups are equal and conclude that the observed 0.292 difference in means is statistically significant. Furthermore, we can reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that the observed difference in median is statistically significant.
Total Productive Maintenance
Based on these results we fail to reject the null hypotheses of the t and z tests that the means of the two TPM analysis groups are equal and conclude that the observed 0.026 difference in means difference is not statistically significant. However, we can reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that there is a statistically significant difference in median.
Just-In-Time
Based on these results we can reject the null hypotheses of the t and z tests that the means of the two JIT analysis groups are equal and conclude that the observed 0.083 difference in means is statistically significant. Additionally, we can reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that there is a statistically significant difference in median.
Basic Elements
Based on these results we can reject the null hypotheses of the t and z tests that the means of the two BE analysis groups are equal and conclude that the observed 0.181 difference in means is statistically significant. We fail to reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that there is not a statistically significant difference in median between these BE groups.
Effective Management System
Based on these results we fail to reject the null hypotheses of the t and z tests that the means of the two EMS analysis groups are equal and conclude that the observed 0.013 difference is not statistically significant. Furthermore, we fail to reject the null hypothesis of the Wilcoxon signed-rank test that the medians of the two groups are equal and conclude that there is not a statistically significant difference in median between these groups.
Figure 4 compares histograms of the before 2009 TQM and after 2009 TQM enabler category analysis groups, for the purpose of visualizing the observed changes. The histograms show a large increase in the proportion of response scored as five and a slight decrease in the responses scored as one through four. This suggests that complete enabler implementation for this category greatly increased and partial enabler implementation was less frequent after the release of the ICH Q10 guidance. It is likely that the trends observed in the complete population results are largely due to the effects of the TQM enabler category.
Figure 4.
Pre 2009 and Post 2009 Total Quality Management (TQM) enabler category histogram comparison
3.4. Analysis by Year
The mean, median and number of observations for all responses from each year that responses were provided are presented in Table 5 below.
Table 5
Yearly Analysis Data
| YEAR | MEAN | MEDIAN | # OF OBSERVATIONS |
|---|---|---|---|
| 2003 | 3.288 | 4 | 8958 |
| 2005 | 3.303 | 3 | 1916 |
| 2006 | 3.247 | 3 | 482 |
| 2007 | 3.169 | 3 | 2342 |
| 2008 | 3.426 | 4 | 2807 |
| 2009 | 3.382 | 4 | 2692 |
| 2010 | 3.362 | 4 | 1727 |
| 2011 | 3.573 | 4 | 1350 |
| 2012 | 3.453 | 4 | 2585 |
| 2013 | 3.258 | 3 | 1593 |
| 2014 | 3.320 | 3 | 5251 |
| 2015 | 3.417 | 4 | 2448 |
| 2016 | 3.411 | 4 | 2899 |
| 2017 | 3.740 | 4 | 1340 |
| 2018 | 3.403 | 4 | 544 |
Open in a separate window
These data were visualized in Online Resource 1, which consist of line graphs of the means and medians vs. the year.
3.5. Discussion
Assessment of Complete Population
Due to the statistically significant differences that were determined in PQS enabler implementation across all enabler categories in the complete population analysis, we propose that ICH Q10 had a positive impact on PQS development and maintenance. The test results confirm an increase in mean and median of all enabler question responses, which is evidence of the population of manufacturing sites in this sample having a greater perceived degree of PQS enabler implementation after the release of the ICH Q10 guidance document. This increase is a result of increasing percentage of responses scored as five and decreasing percentage of responses scored as one or two between the Pre-09 and Post-09 groups, as visualized in Figure 1.
The yearly analysis of the complete population supports the findings that the mean and median of the enabler implementation responses is greater after 2009 than they were before 2009. The lowest mean before 2009 occurred in 2007 with a value of 3.169 and the highest mean occurred in 2008 with a value of 3.426. This increase from 2007 to 2008 represents the second largest increase between years. After 2009, the lowest mean occurred in 2013 with a value of 3.258 and the highest mean occurred in 2017 at value of 3.740. These mean minimums are reflected in the trends of the medians, which generally held at 4 but dipped to 3 from 2005 to 2007 and from 2013 to 2014. The large degree in variability from year to year for the means suggests that changes in enabler implementation may not be solely attributed to quality improvement over time, but instead may be a result of additional variables. In particular, the peak in 2011 is followed by two years of decreasing means and several more years of relatively similar means. However, the large degree of difference in the number of observations in each year is a limitation to this analysis. This fluctuation in observation amount results in continuous introduction and removal of observations and thus difficulty in identifying a longitudinal trend.
Assessment of Enabler Categories
While the complete population results indicate that the ICH Q10 guidance was effective, the further analysis at the enabler category level is essential to determining the impact of ICH Q10 on various PQS elements. This analysis provides insight into the most effective components of ICH Q10 as well as identification of potential areas for further improvement. As mentioned earlier, the PQS is based on a systems approach and each of its components should be evaluated through a holistic approach, taking into account the interactions and effects that changes in other elements could have on individual components of the quality system. Consequently, the following conclusions recognize that each component is important to a complete and effective system and that any changes observed may have been the results of unknown or unanalyzed interactions.
Two enabler categories, TQM and JIT, were determined to have a statistically significant difference in mean and median between the before 2009 and after 2009 analysis groups by the tests performed. The Total Quality Management enabler category displayed the largest difference in mean of all the enabler categories, at a difference of 0.292 compared to the 0.113 difference observed for the complete population. This suggests that TQM enabler implementation improved the most from the release of the ICH Q10 guidance document. Therefore, the ICH Q10 PQS element that can be identified to have been the most effective is the Process Performance and Product Quality Monitoring System.
The other enabler category that was determined to have significant differences in mean and median, Just-In-Time, does not have an ICH Q10 PQS element that can be directly associated with it. However, some aspects of JIT can be attributed to activities performed under the Management Review element. This PQS element is responsible for improving the manufacturing processes and the reallocation of resources to better fit the process, based on the review of regulatory inspections, audits, and periodic quality reviews [(6)]. These responsibilities coincide with the JIT’s specific goals of continual elimination of waste, improvement of product quality, and maximization of production efficiency. Furthermore, the JIT enabler subcategories Layout Optimization and Planning Adherence can be directly improved through the Management Review Process. Therefore, our results suggest that the Management Review ICH 10 PQS element was an effective part of the ICH Q10 guidance as a whole and contributed to the observed improvement in JIT enabler implementation. In addition to the direct impact of the ICH Q10 release on JIT enabler implementation, improvements would have been expected to be observed as a result of maturing PQS behavior. Some improvement could be attributed to this as one of the PQSs main purpose is to change habits and routines in the manufacturing process that would have an impact on JIT behaviors.
In addition to the two enabler categories that were determined to have statistically significant differences in mean and median by all tests, two enabler categories were determined to have a statistically significant difference in mean or median by one test result and not the other. These split results suggest that the observed difference is not as strongly significant and could indicate that some of the test assumptions were not met. These enabler categories are BE and TPM. The BE enabler category includes a collection of practices that are shared by all three technical categories (TPM, TQM and JIT) [(10)]. These practices include the implementation of fundamental OPEX practices like Standardization and Simplification, as well as Visual Management. The implementation of these practices is guided by the Change Management PQS element under the ICH Q10 PQS model. Therefore, the effectiveness of the Change Management portion of the ICH Q10 guidance document can be evaluated through the changes in the BE enabler category. For this enabler category, the difference in means between the two sample groups was found to be 0.181 and this difference was determined to be statistically significant by both the Z and T tests. However, the Wilcoxon Sign Rank test determined there was not a statistically significant difference in median. Based on these results, we believe that the Change Management PQS element was an effective part of the ICH Q10 document and contributed to the observed increase in BE enabler implementation.
TPM is a comprehensive approach to equipment maintenance that emphasizes proactive and preventative maintenance [(12)]. The TPM enabler category is designed to evaluate methods that ensure a high level of equipment stability and availability, which is reflected by the three subcategories Preventative Maintenance, Housekeeping, and Effective Technology Usage [(10)]. Although TPM is a manufacturing philosophy and there is no direct comparison to any of the ICH Q10 PQS elements, there are some enablers in this category that could be covered by Management Review as this includes improving the manufacturing processes. This supports the Management Review ICH 10 PQS element being an effective part of the ICH Q10 guidance, as well as contributing to the observed improvement in TPM enabler implementation. The observed improvement is supported by the Wilcoxon Sign Rank test, which determined that there was a statistically significant difference in median for this enabler category. However, the Z ad T test results did not confirm a statistically significant difference in means for the observed 0.026 increase in mean for the TPM enabler.
Lastly, the EMS enabler category was found to not have a statistically significant difference in mean or median between the prior to 2009 and after 2009 sample groups. Additionally, this enabler category is the only one that showed a decrease in mean from before the ICH Q10 release to after its release. The EMS enabler category was designed to assess the management systems and capabilities of the manufacturing sites. Therefore, it can be reasonably associated with the Management Responsibilities ICH Q10 PQS element. Based on the statistical test results, it appears that the Management Responsibilities were not effective in improving EMS enabler implementation. This suggests that this PQS element was not effectively described in the ICH Q10 guidance document, which contradicts the general consensus that management commitment is crucial for PQS effectiveness. An explanation for this finding is that the Management Responsibilities elements were already fully developed in the PQSs of surveyed pharmaceutical manufacturers. This is supported by the EMS enabler category having the highest mean for both the prior to 2009 and after 2009 sample groups, despite not increasing with the release of ICH Q10. Another potential explanation is that the individuals completing the self-assessment are those responsible for the Management Responsibilities element and the findings are being confounded by bias.
3.6. Conclusion
This study determines if the ICH “Q10 Pharmaceutical Quality Systems” guidance document has had a statistically significant positive impact on the PQS of manufacturing sites around the world. This was determined by the evaluation of the degree of difference in production principles and observable behavior between manufacturing sites prior to ICH Q10 publication and sites after ICH Q10 publication using data from the St. Gallen OPEX Data Benchmarking Questionnaire database. Additionally, this study assesses the statistical significance and degree of difference in the means and medians of each of the five enabler categories. These enabler categories are linked to associated ICH Q10 PQS elements, and the effectiveness of these elements are evaluated.
The results from this study demonstrate that the manufacturing sites studied showed a greater degree of PQS enabler implementation after the release of the ICH Q10 guidance document. Furthermore, 4 of the 5 enabler categories displayed some degree of statistically significant difference in measures of central tendency. While the observed differences in mean and median do not imply causation, the results of this study suggest the implementation of new guidelines had a positive impact. Further study of the implementation of these guidelines through observation at local levels may provide additional insight into which specific changes in routines, processes and behaviors yielded the most tangible results
These findings from this study also suggest that the Management Review, Change Management System, and Process Performance and Product Quality Monitoring System ICH Q10 PQS elements were all effectively described in the guidance and implemented by the manufacturing sites with one exception. The EMS enabler category showed a decrease in mean enabler score, which suggests the Management Responsibilities ICH Q10 PQS element was not effectively described or implemented. Further exploration of this exception may yield detailed insight into the lack of impact.
cGMP issues and inadequacies that can be addressed by an effective PQS continue to be observed by regulatory authorities and thus continued work by industry and regulators is required. The findings of this study provide insight into the effectiveness of the ICH Q10 guidance that may aid in the application of these guidelines to good manufacturing processes. This study also suggests that clarifying and promoting the implementation of the Management Responsibilities, such as resource management and internal communication processes, is one area of improvement identified by this research.
Overall, the ICH Q10 guidance appears to have been effective in whole and across most of its individual PQS elements. Therefore, this study suggests that application of these guidelines continue alongside additional research of its impact at the local level to identify its most tangible impact of cGMP.
Supplementary Material
1761423_Sup_info
Click here to view.(102K, pdf)
Acknowledgements
The authors would like to thank Professor Thomas Friedli and his co-workers at the University of St. Gallen, Institute of Technology Management for access to the OPEX benchmarking survey results.
Funding Statement – This work was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR002378. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
Conflict of Interest Statement – The authors have no conflicts of interest related to this work.
References
1. U.S Food and Drug Administration. Pharmaceutical cGMPs for the 21st Century - A Risk-Based Approach Final Report. In: Services DoHaH, editor. September27, 2004. [Google Scholar]
2. Damron CFDA 483 Observations and Warning Letter Trends. Crowell & Moring; 2019. [Google Scholar]
3. Center for Devices and Radiological Health (CDRH). Quality System (QS) Regulation/Medical Device Good Manufacturing Practices 2018. 02/06/21. Available from: https://www.fda.gov/medical-devices/postmarket-requirements-devices/quality-system-qs-regulationmedical-device-good-manufacturing-practices.
4. International Conference on Harmonisation. ICH Q8 PHARMACEUTICAL DEVELOPMENT. 2009.
5. US Food and Drug Administration. Guidance for Industry: Quality Systems Approach to Pharmaceutical CGMP Regulations. In: U.S. Department of Health and Human Services, editor. 2006. [Google Scholar]
6. International Conference on Harmonisation. ICH Q10 PHARMACEUTICAL QUALITY SYSTEM. 2008.
7. Friedli T, Koehler S, Buess P, Basu P, Calnan N. FDA Quality Metrics Research Final Report Year 12017.
8. García-Bernal J, Ramírez-Alesón M. Why and How TQM Leads to Performance Improvements. Quality Management Journal. 2015;22(3):23–37. [Google Scholar]
9. Cheng TC, Podolsky S. Just-in-Time Manufacturing: An introduction: Springer Netherlands; 1996.
10. Friedli T, Koehler S, Buess P, Eich S, Basu P, Calnan N. FDA Quality Metrics Research Final Report Year 22018.
11. Friedli T, Basu P, Bellm D, Werani J, editors. Leading Pharmaceutical Operational Excellence. 1 ed: Springer-Verlag; Berlin Heidelberg; 2013. [Google Scholar]
12. Vorne. TPM (Total Productive Maintenance)2019. Available from: https://www.leanproduction.com/tpm.html.
