Figures
Abstract
Surgical safety during vacation periods may be influenced by the interplay of several factors, including workers' leave, hospital activity, climate, and the variety of patient cases. This study aimed to highlight an annually recurring peak of surgical mortality during summer in France and explore its main predictors. We selected all elective of open surgical procedures performed in French hospitals between 2007 and 2012. Surgical mortality variation was analyzed over time in relation to workers leaving on vacation, the volume of procedures performed by hospitals, and temperature changes. We ran a multilevel logistic regression for exploring the determinants of surgical mortality, taking into account the clustering of patients within hospitals and adjusting for patient and hospital characteristics. A total of 609 French hospitals had 8,926,120 discharges related to open elective surgery. During 6 years, we found a recurring mortality peak of 1.15% (95% CI 1.09–1.20) in August compared with 0.81% (0.79–0.82, p<.001) in other months. The incidence of worker vacation was 43.0% (38.9–47.2) in August compared with 7.3% (4.6–10.1, p<.001) in other months. Hospital activity decreased substantially in August (78,126 inpatient stays, 75,298–80,954) in relation to other months (128,142, 125,697–130,586, p<.001). After adjusting for all covariates, we found an "August effect" reflecting a higher risk to patients undergoing operations at this time (OR 1.16, 95% CI 1.12–1.19, p<.001). The main study limitation was the absence of data linkage between surgical staffing and mortality at the hospital level. The observed, recurring mortality peak in August raises questions about how to maintain hospital activity and optimal staffing through better regulation of human activities.
Citation: Caillet P, Payet C, Polazzi S, Carty MJ, Lifante J-C, Duclos A (2015) Increased Mortality for Elective Surgery during Summer Vacation: A Longitudinal Analysis of Nationwide Data. PLoS ONE 10(9): e0137754. https://doi.org/10.1371/journal.pone.0137754
Editor: Takeru Abe, Yokohama City University, JAPAN
Received: April 13, 2015; Accepted: August 20, 2015; Published: September 25, 2015
Copyright: © 2015 Caillet et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Data are available from the French Agence technique de l'information sur l'hospitalisation (ATIH Institutional Data Access) for researchers who meet the criteria for access to confidential data. These are edicted by the French national commission governing the application of data privacy laws (the Commission Nationale Informatique et Libertés”). Consequently, data are available after obtaining legal authorization at http://www.atih.sante.fr/bases-de-donnees/commande-de-bases. For a foreign researcher, requests must be sent to: M. Housseyni HOLLA, Directeur de l’ATIH (demande_base@atih.sante.fr).
Funding: PC, CP, JCL, SP and AD received financial support from Hospices Civils de Lyon teaching hospital for independently conducting the submitted work. MJC received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Although more than 230 million major procedures are undertaken worldwide each year [1], the death risk for inpatient surgery remains unexpectedly high [2]. Surgical mortality varies over time within individual hospitals, suggesting that there is opportunity for improvement. Part of this variability is from occasional safety issues, provoking a momentary mortality peak in particular institutions [3]. Another component may relate to the common patterns of seasonality across institutions, as reflected through the recurring death peaks in many hospitals during the same period every year. For example, there is a well-known annual July effect in teaching hospitals at the beginning of a new academic year. Increased surgical mortality is correlated to the influx of inexperienced trainees and the lack of attending oversight, which is possibly related to vacation schedules [4]. Furthermore, there is a higher risk of death for patients who undergo elective surgical procedures later in the working week and on the weekend [5,6].
Among the members of the Organization for Economic Cooperation and Development [7], France is the most generous country with respect to the legal right to paid leave [8]. Workers have six weeks of paid vacation annually on average, and it is common for them to take their longest annual holiday in summer [9]. Due to this recurring annual phenomenon, there may be a risk for patients undergoing surgery in institutions with reduced capacity. Vacation periods can place surgical teams under great pressure and alter the quality of surgery [10]. Inadequate staffing [11] and the inability to maintain sufficient procedural volumes may transiently jeopardise surgical outcomes [12]. High ambient temperatures may also lead to increased mortality [13] as well as cluster patients with poor prognoses and risky comorbidities whose care cannot be delayed.
We hypothesised there might be an “August effect” in France that is characterised by an annually recurring peak in surgical mortality. Using nationwide databases, we aimed to determine its main predictors among vacation periods, hospital activity, climate factors, and the patient case mix.
Materials and Methods
Data
We designed a simultaneous longitudinal analysis of the following three data sources: the nationwide hospital database in acute care, the continuous employment survey among workers, and the national weather newsletter. All elective open surgical procedures performed in French hospitals were selected. The mortality variation was interpreted over time in relation to workers leaving on holiday, the volume of procedures performed within hospitals, and temperature changes.
The nationwide hospital database was obtained for patients discharged over a 6-year period, from January 2007 through December 2012. This database includes information about all inpatient stays in every public and private hospital with acceptable validity [14]. For example, in 2010, the French population totalled 62,765,235 persons [15], resulting in 16,709,216 acute hospitalisations [16]. The standard discharge abstracts for each of these hospitalisations contain compulsory information about the patient (e.g., gender and age) and the patient’s primary and secondary diagnoses, using the International Classification of Diseases, 10th revision (ICD-10 codes), as well as procedural codes associated with the care provided.
Data about the workers’ vacations were obtained from the National Institute of Statistics and Economic Studies in the form of a continuous employment survey aimed at observing the situation of the French work market as part of the European Union labour force survey. Starting in 2001, this survey has continuously logged the work status of 8,000 new people every week, aged from 15 to 75 years old [9].
Climate data were provided by the national weather service. These data are gathered from a network of meteorological stations that are scattered all over the French territory. To monitor seasonal temperature variations, the mean of the maximum daily temperatures was calculated monthly in a hundred geographical areas [17]. Depending on its location, every hospital was then linked to a single geographic area with the corresponding set of temperatures over the study period.
Population
From the French hospital database, all adults who were 18 years of age or older and who underwent open surgery and possibly died during their hospitalisation were identified. To preserve the homogeneity of the sample as well as exclude unavoidable deaths, ophthalmologic, dental, and obstetric procedures were not selected; we also excluded paediatric, ambulatory and palliative care, and organ retrieval cases. Because outpatient surgery is associated with a very low rate of mortality, we further restricted our analysis to inpatient stays with at least one night spent in the hospital. According to the above criteria, 12,793,516 stays with surgical procedures performed in 1,330 hospitals were extracted. We then identified elective procedures that reflected all surgical specialties to enhance the generalisability of the results across the spectrum of surgical care and hospitals. We chose to exclude high-risk emergency cases from the dataset because the corresponding flow of inpatient admissions could not be properly regulated. Finally, given our interest in longitudinally monitoring surgical mortality, we further limited the study population to hospitals that provided continuous care over 72 months, where at least one death was recorded per year, and with at least 100 surgical procedures per year, over the six-year period. Therefore, we were expecting to minimise artefacts from systematic coding errors or nonadjustable case-mix variations. With these exclusions, our final study cohort consisted of 8,926,120 inpatient stays in 609 facilities.
Variables
Our primary outcome measure was in-hospital death within 30 days of patient admission. To adjust mortality for case-mix variations, our set of covariates included the patients’ age, gender, and 31 coexisting conditions extracted from the Elixhauser list of comorbidities [18]. For every procedure, the anatomical site of the surgery was also considered and complexity index was controlled for based on the level of human and material resources involved in performing the procedure.
Additionally, to account for structural or environmental determinants, several variables were considered at the hospital level, including the facility type (public, private, or teaching), volume of surgical procedures performed monthly, and geographic location (North West, North East, South West, South East, or Paris). To measure the potential activity reduction in August at every hospital, we first calculated the expected volume of inpatient stays in August based on the yearly hospital activity, taking into account the number of calendar days in August. Activity reduction was defined as when the observed volume of stays in August was lower than the 95% confidence interval (95% CI) of the expected volume of stays.
Statistical analysis
Data manipulation and analyses were performed using SAS software (version 9.2; SAS Institute Inc., Cary, NC). The month was selected as the unit of temporal analysis because it was the information with utmost granularity for marking the time of patient discharge from hospital. The seasonal patterns among the surgical mortality, workers' vacation rate, hospital activity, and maximum temperatures were plotted over time. In interpreting the variation in these metrics, we were expecting to explore potential correlations and determine whether there were any time-dependent effects. Categorical variables were presented using absolute and relative frequencies, and they were compared between August and other months using the χ2 test. Continuous variables were presented using the means and standard deviation, and they were compared using the Mann-Whitney test. Estimates were accompanied with the corresponding 95% CI.
To explore the determinants of surgical mortality, we ran multilevel logistic regressions (PROC GLIMMIX), taking into account the clustering effect of patients within hospitals. Death was the outcome of interest in the final model, whereas August was the main predictor, which was entered as either a binary variable (August vs. other months) or categorical variable, depending on the activity reduction at each hospital (August with activity reduction or without activity reduction vs. other months). To control for potential confounders, the patient characteristics (e.g., sex, age, Elixhauser comorbidities, surgical procedure codes, and complexity) were selected a priori as clinically important covariates as well as the year of hospital discharge to consider secular trends and coding variations. Additionally, the outcomes were adjusted for the hospital status and location to account for the case-mix and temperature variations across institutions. Interactions between the main predictor and other covariates entered in the model were tested. The results were presented as adjusted odds ratios (OR) with corresponding 95% CI. Furthermore, a sensitivity analysis was performed on primary outcome measure to account for potential variations in estimates by extending mortality measure periods from 30 to 45 days after patient admission.
Results
Characteristics of the patients and hospitals
A total of 609 French hospitals had 8,926,120 open elective surgical discharges between 2007 and 2012 (Fig 1). The hospital and inpatient stay characteristics are shown in Table 1. There were 73,477 deaths, accounting for an overall perioperative mortality of 0.8%, ranging from 0.1% to 3.0% per institution. Hospitals demonstrated an average of 237 admissions per month during the study period, and the mean temperature recorded in their respective geographic area was 16.9 degrees Celsius (°C).
Flow diagram of hospitals and stays selected for inclusion.
Seasonal patterns
Monthly variations in the surgical mortality rate were highly reproducible across years and demonstrated a recurring summer mortality peak (Fig 2a). Across the 6 years studied, the mortality rate in August was 1.15% (95% CI, 1.09–1.20), while it was 0.95% (95% CI, 0.89–1.01, p = .004) in July and 0.81% (95% CI, 0.79–0.82, p<.001) in the other months. Overall, 5,378 deaths occurred in August, representing 1,604 excess deaths (42.5%; 1,604/3,774) relative to what would be expected based on the other months' average rates. Additionally, a mortality peak was observed across the spectrum of various surgical specialties (Fig 2b) and institutions (Fig 2c), whatever the baseline risk of mortality for the patients.
These figures present the monthly change in crude mortality rate by year (2a), surgical specialty (2b) and quartile of hospital mortality (2c).
We observed the same seasonal patterns regarding workers' leave, mirroring the surgical mortality variations (Fig 3a). The rate of workers on vacation was 43.0% (95% CI, 38.9–47.2) in August, compared with 19.3% (95% CI, 16.3–22.4, p<.001) in July and 7.3% (95% CI, 4.6–10.1, p<.001) in the other months (Fig 3b). In parallel, hospital activity (Fig 3c) decreased substantially in August (78,126 inpatient stays in average; 95%CI, 75,298–80,954) compared to July (110,817; 95% CI, 106,727–114,907, p = .004) and the other months (128,142; 95CI% 125,697–130,586, p<.001). With respect to temperature (Fig 3d), the warmer months were August (25.6°C; CI 95%, 25.4–25.9) and July (25.2°C; 95%CI, 24.9–25.4) compared with the other months (16.1°C; 15.9–16.2, p<.001).
The figures present the monthly changes in mortality rates (3a), along with those of workers' leave (3b), hospital activity (3c) and maximum temperatures (3d).
“August effect”
Among all patients hospitalised for elective surgery, 468,758 (5.3%) were admitted in August, including 422,117 (90.1%) stays in hospitals with activity reduction and 46,641 (9.9%) stays in hospitals without activity reduction. There were fewer women and musculoskeletal procedures in August, while digestive and integumentary surgeries were more frequent (Table 2). Furthermore, the market share for private hospitals was reduced in August relative to that of teaching and public institutions.
Based on a multivariate analysis, there was a reduction in the risk of postoperative death from 2007 through 2012 (Table 3). Patient characteristics that were independently associated with death included male gender, increasing age, Elixhauser comorbidities, and surgical procedure complexity. The riskier procedures were operations on the respiratory, digestive, nervous, or cardiovascular systems. After adjusting for all covariates, we found an “August effect”, reflecting a higher risk for patients undergoing operations at this time (OR 1.16; 95% CI, 1.12–1.19, p<.001). Additionally, hospital activity reduction in August was associated with increased mortality (OR ranging from 1.15 to 1.36, p<.001), while death risk in hospitals without activity reduction was not significant (OR 1.06; 95% CI, 0.97–1.16). On the other hand, the hospital geographic location did not influence the occurrence of inpatient death in August compared with other months. We found interactions between the variables entered in the model related to sex, comorbidities, age and operations on the hematologic, lymphatic, cardiovascular, digestive, and musculoskeletal system. The “August effect” persisted in all corresponding strata and by extending the mortality period to 45 days after patient admission (see S1 Fig and S1 Table).
Discussion
We found an increased risk of death for patients in France who undergo elective surgery during August, evidenced through a recurring annual peak in mortality across the main surgical specialties. This pattern would result from two primary factors, after simultaneously adjusting the outcomes for the patient, procedure and institution characteristics.
First, temporary understaffing due to massive holiday departure can cause an excess in hospital mortality if it is not properly managed. Since 1969, French workers have been allowed to take four weeks of paid leave per year. Currently, they are entitled to 30 vacation days annually, which is more than in most European countries [8]. What is remarkable is the concentration of their leave and the potential issue that traditionally a majority of the French population takes a full month of vacation during summer, particularly in August [9]. As a consequence, inadequate matching of hospital staff with patient needs can jeopardise care safety [11] and overwhelm the remaining health personnel [19]. Although recruitment of interim staff represents a flexible alternative for tackling momentary staff reduction, this solution is expensive and can disrupt regular team dynamics as interim healthcare workers are unfamiliar with a particular hospital's context. Especially, an unexpectedly high turnover rate of operating room members during this period may impact the surgical outcomes [20]. Compared with other countries, this does not necessarily reflect trainees rotating or newly qualified doctors moving to new posts, which normally occurs in November in the French institutions [21]. However, it appears plausible that the most experienced surgeons are more likely to take their summer leave in August than their youngest colleagues, leading to suboptimal care at this period.
Second, we found that patients who underwent surgery in hospitals with significant activity reduction in August presented with higher mortality. The more the hospital volume of procedures decreased in August, the more the patients were at risk of dying. A potential explanation is that those institutions unable to maintain a regular volume of surgical procedures over the year would not provide safe care at any time. Slowing down activity below a critical threshold [12], in conjunction with bed closure and temporary staff reduction during summer vacation, may disorganise the clinical pathways. In turn, this may induce life-threatening situations and impair the capacity of surgical teams or intensive care units for appropriately managing harmed patients [22].
Although our analysis considered climate factors, similar patterns in seasonal mortality were evidenced across the years, and the repetition of the “August effect” was not influenced by anomalous events [3] such as occasional heat waves. In addition, we observed an increased mortality in August relative to July, while there were equivalent temperatures over these months. Similarly, there was no effect related to the hospitals’ geographic location based on multivariate modelling in spite of the well-established temperature gradient from the north to south of France during the summer. Therefore, we consider that the hospital mortality increase was not related to high temperatures in August [23], which seems consistent for elective surgery performed in an air-conditioned environment.
In conducting a large population-based study and observing the repetition of significant seasonal patterns over the years, we acknowledge several limitations to our study. We were not able to control for provider characteristics that impact the outcomes, such as the team composition or surgeon’s experience [20, 24]. Accurate data on the surgical staff available at the time of surgery would be a step to highlighting the direct link between personnel staffing and patient mortality at the hospital level. Furthermore, we monitored the in-hospital mortality over a broad range of procedures without knowing the prognosis of patients outside the hospital or separating preventable from inevitable deaths [25]. This point is critical, as we assumed that hospital-wide mortality was a valid metric of patient safety [26]. Risk adjustment can only account for factors that can be identified and measured accurately [27]. Although we considered the patient characteristics and procedural complexity in the adjustment scheme, this may not be sufficient to ascertain the impact of the differential case mix on the surgical outcome. “Do not resuscitate” designations and detailed information about the presence of diagnoses on admission were not available in the administrative data for adjusting mortality [28]. Emergency admissions were excluded from our initial dataset, but there was no means for considering quasi-emergency cases, namely high-risk patients requiring early surgical treatment. Accordingly, reductions in the hospital procedural volumes during summer may reflect the postponement of true elective cases until after the holidays; as a result, there is a preponderance of semi-urgent cases with a poorer prognosis performed in the August timeframe. However, inpatient mortality was adjusted for several risk factors (e.g., cancer) and we observed the same seasonal pattern across a broad range of surgical specialties and hospitals.
Despite observing a reduction in the risk of postoperative death over the years [29], the recurring August mortality peak in France raises important questions about how to maintain hospital activity and optimal staffing levels. Mandatory measures have been implemented to deal with hazardous situations. Although these threats to population health have a very low probability of occurrence, every hospital has to develop plans for managing crisis situations, such as epidemic outbreaks, natural disasters, terrorism attacks or other major accidents [30]. Conversely, poor awareness of common safety issues that can be spotted by repeated patterns over time may be associated with a lack of interest from health authorities. Such seasonal disruptions in care process are predictable every year and could be regulated through better coordination of human activities. Reductions in the volume of elective surgery may result from vacation leaves among patients and healthcare workers. Both hospital staff and patients prefer to have care coordinated outside of the August summer vacation time period. Rather than proposing the unpopular solution of suspending annual leaves in August, a fairly straightforward way of eradicating this problem would consist of balancing vacations over the course of the year. On the condition of overcoming both cultural and political barriers, resolving the “August effect” will require a smoothing of the volume of surgical procedures over months and a more rational scheduling of worker vacations.
Supporting Information
S1 Fig. Sensitivity analysis on primary outcome.
Seasonal pattern of surgical mortality at 30 days and 45 days.
https://doi.org/10.1371/journal.pone.0137754.s001
(TIF)
S1 Table. Sensitivity analysis on primary outcome.
Factors independently associated with in-hospital death within 30 days and 45 days after patient admission.
https://doi.org/10.1371/journal.pone.0137754.s002
(DOCX)
Acknowledgments
The authors are grateful to Nicolas Voirin (Lyon) for his helpful comments on the statistical analysis. This work was presented to the 2014 International Conference of the International Society for Quality in Health Care (ISQua) at Rio de Janeiro and to the 2014 Clinical Congress of the American College of Surgeons (ACS) at San Francisco.
Ethics approval: Waived. Our national dataset, in accordance to the laws that regulate hospital database in France, namely articles L. 6113–7 et L. 6113–8 of the Public Health Code, was anonymized by hospitals and ATIH (French agency of hospital information). Since the study was strictly observational and used anonymous data, in accordance to the laws that regulate “non-interventional clinical research” in France, namely articles L.1121-1 and R.1121-2 of the Public Health Code, did not require the written informed consent from the participants or the authorization from any other ethics committee to conduct this survey.
Author Contributions
Conceived and designed the experiments: PC CP JCL MJC SP AD. Performed the experiments: PC CP SP AD. Analyzed the data: PC CP JCL MJC SP AD. Contributed reagents/materials/analysis tools: PC CP JCL MJC SP AD. Wrote the paper: PC CP JCL MJC SP AD.
References
- 1. Weiser TG, Regenbogen SE, Thompson KD; Haynes AB, Lipsitz SR, Berry WR, et al. An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet. 2008;372(9633):139–44. pmid:18582931
- 2. Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C, et al. Mortality after surgery in Europe: a 7 day cohort study. Lancet. 2012;380(9847):1059–65. pmid:22998715
- 3. Duclos A, Polazzi S, Lipsitz SR, Couray-Targe S, Gawande AA, Colin C, et al. Temporal variation in surgical mortality within French hospitals. Med Care. 2013;51(12):1085–93. pmid:24231784
- 4. Englesbe MJ, Pelletier SJ, Magee JC, Gauger P, Schifftner T, Henderson WG, et al. Seasonal variation in surgical outcomes as measured by the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP). Ann Surg. 2007;246(3):456–62; discussion 463–5. pmid:17717449
- 5. Aylin P, Alexandrescu R, Jen MH, Mayer EK, Bottle A. Day of week of procedure and 30 day mortality for elective surgery: retrospective analysis of hospital episode statistics. BMJ. 2013;346:f2424. pmid:23716356
- 6. Zare MM, Itani KM, Schifftner TL, Henderson WG, Khuri SF. Mortality after nonemergent major surgery performed on Friday versus Monday through Wednesday. Ann Surg. 2007 Nov;246(5):866–74. pmid:17968181
- 7.
Organisation for Economic Co-operation and Development (OECD) [Internet]. Available: http://www.oecd.org. Accessed 17 July 2015.
- 8.
Ray R, Sanes M, Schmitt J. No-vacation nation revisited. 2013. [CEPR web Site]. Available: http://www.cepr.net/index.php/publications/reports/no-vacation-nation-2013. Accessed 10 April 2015.
- 9.
Biausque V, Thévenot C. [In 2010, employees took an average of six weeks off]. INSEE Première. 2012;(1422). Available: http://www.insee.fr/fr/ffc/ipweb/ip1422/ip1422.pdf. Accessed 10 April 2015.
- 10. Duclos A, Touzet S, Soardo P, Colin C, Peix JL, Lifante JC. Quality monitoring in thyroid surgery using the Shewhart control chart. Br J Surg. 2009;96(2):171–74. pmid:19160350
- 11. Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, Harris M. Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011;364(11):1037–45. pmid:21410372
- 12. Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002;346(15):1128–37. pmid:11948273
- 13. Basu R. High ambient temperature and mortality: a review of epidemiologic studies from 2001 to 2008. Environ Health. 2009;8:40. pmid:19758453
- 14. Couris CM, Polazzi S, Olive F, Remontet L, Bossard N, Gomez F, et al. Breast cancer incidence using administrative data: correction with sensitivity and specificity. J Clin Epidemiol. 2009;62(6):660–666. pmid:19070463
- 15.
National Institute of Statistics and Economic Studies (INSEE). [INSEE Web Site]. National census of the resident population of France in 2010. Available: http://www.insee.fr/fr/publications-et-services/default.asp?page=dossiers_web/population/population_francaise.htm. Accessed 10 April 2015.
- 16.
Technical Agency for Hospital Information (ATIH). [ATIH Web Site]. French hospitals episode statistics in 2010. Available: http://stats.atih.sante.fr/cgi-bin/broker?annee=2010&base=0&typett=1&ok=Lancer+le+traitement&_program=mcoprog.affiche_present.sas&_service=default&_debug=0&tab=1. Accessed 10 April 2015.
- 17.
French National Weather Information Service (Meteo France). [Meteo France Web Site] Monthly climate newsletter from the national weather. Available: https://donneespubliques.meteofrance.fr/?fond=produit&id_produit=129&id_rubrique=29. Accessed 10 April 2015.
- 18. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–39. pmid:16224307
- 19. Wallace JE, Lemaire JB, Ghali WA. Physician wellness: a missing quality indicator. Lancet. 2009;374(9702):1714–21. pmid:19914516
- 20. Xu R, Carty MJ, Orgill DP, Lipsitz SR, Duclos A. The teaming curve: a longitudinal study of the influence of surgical team familiarity on operative time. Ann Surg. 2013;258(6):953–57. pmid:23407297
- 21. Haller G, Myles PS, Taffé P, Perneger TV, Wu CL. Rate of undesirable events at beginning of academic year: retrospective cohort study. BMJ. 2009;339:b3974. pmid:19826176
- 22. Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, et al. Challenges in evaluating surgical innovation. Lancet. 2009;374(9695):1097–1104. pmid:19782875
- 23. Bouchama A, Knochel JP. Heat stroke. N Engl J Med. 2002;346(25):1978–88. pmid:12075060
- 24. Duclos A, Peix JL, Colin C, Kraimps JL, Menegaux F, Pattou F, et al. Influence of experience on performance of individual surgeons in thyroid surgery: prospective cross sectional multicentre study. BMJ. 2012;344:d8041. pmid:22236412
- 25. Lilford R, Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ. 2010;340:c2016. pmid:20406861
- 26. Shahian DM, Wolf RE, Iezzoni LI, Kirle L, Normand S-LT. Variability in the measurement of hospital-wide mortality rates. N Engl J Med. 2010;363(26):2530–39. pmid:21175315
- 27. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet. 2004;363(9415):1147–54. pmid:15064036
- 28. Pine M, Jordan HS, Elixhauser A, Fry DE, Hoaglin DC, Jones B, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA. 2007;297(1):71–76. pmid:17200477
- 29. Bainbridge D, Martin J, Arango M, Cheng D. Evidence-based Peri-operative Clinical Outcomes Research (EPiCOR) Group. Perioperative and anaesthetic-related mortality in developed and developing countries: a systematic review and meta-analysis. Lancet. 2012;380(9847):1075–81. pmid:22998717
- 30.
French Ministry of Health. White plan and crisis management: A guide to design white plan within health facilities. 2006. Available: http://www.sante.gouv.fr/IMG/pdf/plan_blanc_2006.pdf. Accessed 10 April 2015.