Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Soluble B and T Lymphocyte Attenuator Correlates to Disease Severity in Sepsis and High Levels Are Associated with an Increased Risk of Mortality

Abstract

Introduction and aims

B- and T-lymphocyte Attenuator (BTLA), Cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) and Programmed Death 1 (PD-1) are co-inhibitory receptors that regulate T cell activation. In the present study of ICU-treated patients we measured plasma concentrations of their soluble isoforms, with the aim to evaluate their potential as sepsis biomarkers and utility as prognostic indicators.

Methods

101 patients with sepsis, 28 patients with non-infectious critical illness (ICU controls) and 31 blood donors (healthy controls, HC) were included in the study. Plasma concentrations of soluble BTLA (sBTLA), CTLA-4 (sCTLA-4) and PD-1 (sPD-1) were measured with ELISA in serial blood samples. Comparisons were made with Mann-Whitney U test and correlations were assessed with Spearman’s Rank correlation test. Cox proportional hazard models, with sBTLA and sPD-1 as fixed and sBTLA as time-varying covariates, were used to determine association with 28-day mortality.

Results

sBTLA levels were significantly higher in the sepsis cohort (median 14 ng/mL, IQR 8–29) compared to ICU controls (9 ng/mL, IQR 5–26, p = 0.048) and HC (2.9 ng/mL, IQR 0.9–9.1, p<0.01), and correlated to SOFA score. sBTLA levels were higher in 28 day sepsis non-survivors than in survivors (baseline median 28 ng/mL, IQR 13–41 vs 13 ng/mL, IQR 8–23, p = 0.04). After adjustment for age and comorbidities, the relative risk of 28 day mortality was nearly 5-fold higher in sepsis patients with a baseline sBTLA > 21 ng/mL, compared to those with a level below this threshold. sBTLA was even more associated with mortality in the time-varying analysis. sPD-1 levels were lower in the sepsis cohort compared to HC but not compared to ICU controls and were not associated with mortality. sCTLA-4 was detectable in only one subject.

Conclusion

Plasma concentrations of soluble BTLA were increased early in sepsis/septic shock and correlated to severity of disease. A baseline concentration >21ng/mL was associated with a poor prognosis.

Citation: Lange A, Sundén-Cullberg J, Magnuson A, Hultgren O (2017) Soluble B and T Lymphocyte Attenuator Correlates to Disease Severity in Sepsis and High Levels Are Associated with an Increased Risk of Mortality. PLoS ONE 12(1): e0169176. https://doi.org/10.1371/journal.pone.0169176

Editor: Abderrezak Bouchama, King Abdullah International Medical Research Center, SAUDI ARABIA

Received: July 20, 2016; Accepted: December 13, 2016; Published: January 5, 2017

Copyright: © 2017 Lange et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data is available, and attached to the submission as an Excel file.

Funding: The study was conducted with the financial support from research grants from the Research Committee Region Örebro County, Nyckelfonden Region Örebro County, the Olle Engkvist fund, the Signe and Olof Wallenius trust, ALF project funding, and the Karolinska University Hospital, Huddinge, Research Foundation.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Despite decreasing mortality rates over the past two decades, severe sepsis and septic shock still belong to the most serious acute medical conditions and are leading causes of death in the ICU [1, 2]. While most patients who make it through the acute phase recover quickly, some enter a prolonged state of organ dysfunction, run a risk of super-infections and often fail to clear infections, a condition referred to as immunoparalysis [3–6]. Adding further complexity to this process, although pro-inflammation often dominates in the early sepsis phase, there is simultaneous anti-inflammatory cytokine activity, and T cell activation and proliferation occurs in parallel with increased apoptosis [7–9].

Antigen-specific activation of naïve T lymphocytes requires double signals; T cell receptor (TCR) recognition of antigen bound to MHC on Antigen presenting cells (APCs), and a second signal delivered through the interaction between the co-receptor CD28 and its cognate ligands CD80 and CD86 on the same APC [10]. The magnitude of the immune response is regulated by an intricate network of co-signaling cell-surface bound receptors and their corresponding ligands. Programmed Death 1 (PD-1), Cytotoxic T Lymphocyte-associated Antigen 4 (CTLA-4) and B- and T-lymphocyte Attenuator (BTLA) are three well characterized co-inhibitory receptors, through which cell activation and proliferation can be abated. CTLA-4 competes with CD28 for interaction with the CD80 and CD86 ligands on APCs, and by interfering with co-stimulation CTLA-4 can have an attenuating effect on T cell activation [11]. PD-1 is expressed on T-cells upon activation, and has two ligands; PD-L2, which is limited to APCs, and PD-L1, which is expressed on various immune cells [12]. BTLA is broadly expressed: on B and T lymphocytes, macrophages, dendritic cells and NK cells and can be either up- or down-regulated after stimulation depending on the cell type [13]. Its ligand Herpes Virus Entry Mediator (HVEM) is also expressed on both sides of the APC-lymphocyte entity. BTLA-HVEM as well as PD-1-PD-L1/PD-L2 interactions may convey an inhibiting signal [12, 14]. Negative co-stimulatory pathways are important both in the defense against infection and in maintaining peripheral tolerance and immune homeostasis, which is illustrated by the fact that mice deficient in PD-1, BTLA or CTLA-4 develop autoimmune and lymphoproliferative disease [15–17]. While increased PD-1 expression on T cells is an established marker of immunoparalysis in sepsis, the roles of CTLA-4 and BTLA remain to be determined [18–20]. BTLA and PD-1 expression was shown to be altered on peripheral blood T cells from septic patients. Increased PD-1 expression has also been suggested as a marker of increased risk for nosocomial infections and of a poor prognosis [18, 21]. Experimental models have demonstrated that blockade of the PD-1 and CTLA-4 pathways can improve outcome in sepsis [22–25].

It has been established that there are soluble isoforms of the co-inhibitory receptors, lacking the transmembrane region, and it was shown in vitro that they can antagonize negative co-signaling by competitively binding the corresponding ligand of their membrane-bound equivalents [26–28]. While a recombinant CTLA fusion protein (abatacept), that interferes with the co-stimulatory signal required for T cell activation, has been used for treatment of rheumatoid arthritis unresponsive to anti TNF treatment since 2005, the biological activity of soluble co-inhibitory receptors still remains to be discovered. To examine their participation in immune regulation, we aimed to measure plasma concentrations of the soluble isoforms of PD-1, CTLA-4 and BTLA in critically ill patients and evaluate their usefulness as sepsis biomarkers and indicators of severity of disease and prognosis.

Methods

Setting and study population

Inclusion criteria were: a diagnosis of severe sepsis or septic shock within the last 24 hours in patients admitted to, or already in, the mixed intensive care unit of Karolinska University Hospital in Huddinge. Severe sepsis and septic shock were defined according to the criteria proposed by the American College of Chest Physicians/Society of Critical Care Medicine [29]. Sepsis was defined as an infection in the presence of a systemic inflammatory reaction (SIRS). SIRS was defined by at least two of the following criteria: 1) body temperature higher than 38°C or lower than 36°C, 2) heart rate higher than 90/min, 3) respiratory rate higher than 20/min or PaCO2 lower than 32 mmHg, 4) white blood cell count higher than 12000 or lower than 4000 cells/μl. Severe sepsis was defined as sepsis associated with organ dysfunction, hypoperfusion or hypotension and septic shock was defined as sepsis with persistent hypotension despite adequate fluid resuscitation.

Control patients were defined as patients admitted to the ICU for non-infectious critical illness within the last 24 hours, and with an expected length of stay in the ICU longer than 24 hours. Exclusion criteria in both cohorts were age <18 years.

A total of 130 patients were enrolled in the study between Oct 2005 and June 2009. After review of complete patient charts and laboratory and microbiological tests, one control patient was reclassified as septic shock and one as severe sepsis; two septic shock patients were reclassified as severe sepsis, and one as control. Finally, one patient was excluded from analysis altogether because a clear classification proved impossible. This left 101 patients with severe sepsis (n = 14) or septic shock (n = 87) (sepsis cohort) and 28 patients who were ICU-treated for other critical conditions (ICU-controls). Patients were also reclassified according to The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) [30]. According to this, sepsis is defined as life-threatening organ dysfunction (defined as an increase of 2 points or more in the Sequential Organ Failure Assessment (SOFA) score) due to a dysregulated host response to infection. Septic shock is defined as sepsis and persistent hypotension (despite adequate volume resuscitation) requiring vasopressors to maintain MAP greater than or equal to 65 mm Hg and a lactate greater than or equal to 2 mmol/L. With Sepsis-3 definitions 23 patients within the sepsis cohort had sepsis and 78 had septic shock. 31 blood donors enrolled between 2010 and 2013 served as healthy control subjects (healthy controls, HC).

Patient characteristics

Clinical data was recorded for study patients, and age and sex for healthy control subjects. Acute Physiology and Chronic health Evaluation (APACHE) score was determined on admittance to the ICU and severity of disease was assessed daily with Sequential Organ Failure Assessment (SOFA) score. Blood and other relevant cultures were performed and the results of these, as well as of biological parameters, were recorded.

Ethics

The study was conducted in accordance with the declaration of Helsinki. It was approved by the regional ethical review board of Stockholm, ref. 217/03. A written informed consent was obtained from study subjects or next of kin.

Plasma samples

Blood samples for routine biochemical tests and plasma for later analyses were collected from study patients at inclusion and at 24, 48, 96 and 144 hours and kept at -80°C until used. Healthy controls were sampled in connection with blood donation and plasma was stored correspondingly.

ELISA analyses

sPD-1 was analyzed at 1:2 dilution with RayBio Human PD-1 ELISA Kit, detection range 24.58–6000 pg/mL, RayBiotech, Inc. sBTLA was analyzed at 1:5 dilution with Human BTLA ELISA Kit, detection range 0.47–30 ng/mL, Cusabio. Plasma sCTLA-4 was analyzed at 1:2 dilution with Human sCTLA-4 Platinum ELISA, detection range 0.16–10 ng/mL, eBioscience. All ELISA analyses were performed according to the manufacturers’ instructions.

Statistical analysis

IBM SPSS Statistics for Windows, Version 22.0. Armonk NY: IBM Corp and Stata Statistical Software: Release 14. College Station, TX: StataCorp LP were used for statistical calculations. Plasma concentrations were summarized with median and interquartile range (IQR), according to data distribution. Comparison between the three study cohorts were made by Kruskal-Wallis test followed by pairwise group Mann-Whitney test corrected for multiple testing in the strong sense with the Bonferroni-Holm method [31]. Subgroups were compared with Mann-Whitney U test. Linear regression with baseline markers on log10 scale as outcome was used to evaluate age-adjusted differences between study groups. Correlations were analyzed with Spearman’s rank correlation test and presented as Spearman’s rho. Logistic regression was used to estimate the ability of biomarkers to diagnose sepsis and to predict mortality, by calculating the area under the Receiver Operating Characteristic (ROC) curve (AUC) of the sensitivity and 1-specificity. Risk of mortality was presented with the Kaplan-Meier method. Subjects were followed from inclusion until day 28 or the day of death, with no censoring. 3 patients were not sampled on the day of inclusion and were not included in the survival analysis. Cox proportional hazard models were used to evaluate if baseline levels of biomarkers were associated with mortality, adjusted for a) age, b) age and baseline comorbidities that were judged to be possible confounders (cardiovascular disease, chronic heart failure, COPD, diabetes, renal failure, liver disease and malignancy) on a categorical scale: 0, 1 and ≥2 comorbidities, due to no events (death) in patients with COPD or liver disease or more than 2 comorbidities, and 3) age, comorbidities and s-lactate. Biomarkers were evaluated on continuous log10 scales, as well as on categorical scales which were created as follows: The baseline concentration of the respective marker in the sepsis cohort was divided into three categories based on the 33rd and 67th percentile. To assess the association between sBTLA as a dynamic marker and mortality, we used a time-varying Cox model with sBTLA as the time-varying risk factor. Due to death, transfer to other ICUs and to medical or surgical wards because of recovery, as well as loss because of hemolysis and missed sampling, the number of plasma samples available for analysis diminished over the study period as demonstrated in (Fig 1).

thumbnail
Download:
Fig 1. Plasma samples available for analysis over the course of the study.

Day of inclusion: Sepsis cohort; 3 missed samples. ICU controls; 2 missed samples. At 24 hours: Sepsis cohort; 4 dead, 4 transferred to other ICU:s and 1 discontinued the study. ICU controls; 1 dead, 1 transferred to another ICU. At 48 hours: Sepsis cohort; 3 transferred to other ICU:s and 4 missed samples. ICU controls; 1 dead, 1 transferred to another ICU and 1 missed sample. At 96 hours: Sepsis cohort; 4 dead, 2 transferred to other ICUs, 2 discontinued and 4 missed samples. ICU controls; 5 dead patients and 5 missed samples. 144 hours: Sepsis cohort; 1 dead, 1 transferred to another ICU, 3 discontinued the study, 4 were discharged from the ICU and 8 missed samples. ICU control; 2 dead patients and 3 missed samples.

https://doi.org/10.1371/journal.pone.0169176.g001

Missing sBTLA data were imputed with multiple imputation (MI) technique implemented in the software Stata, based on the concepts of Rubin [32]. Day of sample, age, sex and baseline data; comorbidities (hypertension, cardiovascular disease, chronic heart failure, COPD, diabetes, renal failure, liver disease and malignancy), immunocompromised or not, acquisition of infection (community or hospital), SOFA-score and APACHE II score were used as predictors for the log sBTLA as well as for the categorical sBTLA imputation model. Association measures were hazard ratios (HR) with 95% confidence intervals (CI). P-values less than 0.05 were considered statistically significant.

Results

Characteristics of the study population

In the sepsis cohort 51.5% had a community-acquired and 48.5% a hospital-acquired infection, defined as onset of infection later than 48 hours after hospitalization, within 72 hours after hospital discharge or within 30 days after surgery. As defined by APACHE II criteria: had received immune-suppressive therapy, chemotherapy, radiation, long term or recent high dose steroids or had a disease sufficiently advanced to suppress resistance to infection, 28% in the sepsis cohort and 4% (one patient) among ICU controls were immunocompromised at baseline [33]. The types of immunosuppression in the sepsis cohort were; chemotherapy (n = 14), immunosuppressive therapy due to solid organ transplant (n = 8), autologous/allogeneic stem cell transplantation (n = 2), immunosuppressive therapy due to other disease (n = 2), neutropenia (n = 1) and HIV infection (n = 1). 44% had recent acute or elective surgery. The most common focus of infection was abdominal (41%) followed by pulmonary (24%) and urinary tract (15%). A positive blood culture was found in 48% of sepsis patients. Among ICU controls the most common diagnosis was cardiac arrest. In this cohort, 43% developed an infection during the study period (median 2.5 days) (Table 1).

thumbnail
Download:
Table 1. Characteristics of the study population.

https://doi.org/10.1371/journal.pone.0169176.t001

Comparison of sBTLA and sPD-1 in septic patients, ICU controls and healthy controls

The Kruskal-Wallis test showed significance for differences between groups for the baseline concentration of sBTLA (p<0.01) and sPD-1 (p<0.01). sBTLA plasma concentrations were highest in the sepsis cohort (median 14 ng/mL, IQR 8–29), significantly lower in ICU controls (median 9 ng/mL, IQR 5–26, p = 0.048) and lowest in HC (median 5 ng/mL, IQR 5–8, compared to the sepsis cohort p<0.01, to ICU controls p = 0.03). Significant differences were still present after age adjustment (Fig 2A).

thumbnail
Download:
Fig 2. Plasma sBTLA and sPD-1 concentrations at baseline in the sepsis cohort, ICU controls and blood donors.

(a) sBTLA concentrations in the sepsis cohort (median 14 ng/mL, IQR 8–29), ICU controls (median 9 ng/mL, IQR 5–26) and HC (median 5 ng/mL, IQR 5–8). (b) sPD-1 concentrations in the sepsis cohort (1.3 ng/mL, IQR 0.6–5.8), ICU controls (2.4 ng/mL, IQR 0.7–5) and HC (2.9 ng/mL, IQR 0.9–9.1). Bars indicate median concentrations. Linear regression with markers on log10 scale as outcome gave the following age-adjusted p-values: sBTLA: sepsis vs HC (p<0.01), ICU controls vs HC (p = 0.02) and sepsis vs ICU controls (p = 0.03). sPD-1: sepsis vs HC (p<0.01), ICU controls vs HC (n.s.) and ICU controls vs sepsis (p = 0.27). P-values in figure and text are adjusted with the Bonferroni-Holm method.

https://doi.org/10.1371/journal.pone.0169176.g002

sPD-1 concentrations were significantly lower in the sepsis cohort compared to HC (1.3 ng/mL, IQR 0.6–5.8, vs 2.9 ng/mL, IQR 0.9–9.1, p = 0.01) but there was no significant difference compared to ICU controls (2.4 ng/mL, IQR 0.7–5, p = 0.2) or between ICU controls and HC (p = 0.17). Significant differences were still present after age adjustment (Fig 2B). In terms of predictive ability to diagnose sepsis in the ICU-treated cohort (sepsis + ICU controls), AUC for sBTLA and sPD-1 were 0.626 (95% CI 0.492–0.76, p = 0.048) and 0.603 (95% 0.477–0.730, n. s) respectively. sCTLA-4 was analyzed in plasma sample series from 16 patients in the sepsis cohort, 4 ICU-controls and 5 blood donors. We found a detectable, but low level of sCTLA-4 in only one sepsis plasma sample (0.8 ng/mL) and in no sample from the other two cohorts. Therefore we discontinued analysis of this marker.

Subgroup analysis

In the sepsis cohort no significant differences in sBTLA or sPD-1 concentrations were observed related to focus of infection, between community- or hospital-acquired infection (p = 0.6 for sBTLA, p = 0.9 for sPD-1 at baseline) or between patients with or without leukopenia (n = 18) (p = 0.5 for sBTLA, p = 0.2 for sPD-1). Sepsis patients who were defined as immunocompromised by APAHE II criteria at the time of study inclusion had significantly higher levels of sBTLA at baseline compared to those who were not (p = 0.048) (Fig 3).

thumbnail
Download:
Fig 3. Baseline sBTLA in sepsis patients defined as immunocompromised or not at baseline.

Immunocompromised (n = 27) (median 20 ng/mL, IQR, 9–55) or not (n = 71) (median 12 ng/mL, IQR 8–23) (defined by APACHE II criteria at baseline).

https://doi.org/10.1371/journal.pone.0169176.g003

There was no significant difference in sPD-1 concentrations in this respect (p = 0.4).

Correlation between sBTLA and sPD-1 and other biomarkers

sBTLA and sPD-1 were not correlated (Spearman’s rho -0.08, p = 0.06). Correlations were analyzed between baseline concentrations of the studied markers and leukocyte count, CRP and lactate levels. In the sepsis cohort sBTLA levels correlated moderately to lactate (Fig 4).

thumbnail
Download:
Fig 4. Correlation between baseline sBTLA and lactate in the sepsis cohort.

Data is presented on log10 scales due to non-Gaussian distribution of both sBTLA and lactate.

https://doi.org/10.1371/journal.pone.0169176.g004

No other significant correlations between sBTLA or sPD-1 and the above-mentioned markers were found.

Markers of severity of disease and prognosis in sepsis

sBTLA levels correlated positively to SOFA-score, moderately at 24 and 48 hours and weakly at other time points, but no correlation was found to APACHE score (rho 0.1, p = 0.3) (Fig 5A–5E).

thumbnail
Download:
Fig 5. Correlation between sBTLA and SOFA score in sepsis patients.

a) baseline b) 24 hours c) 48 hours d) 96 hours and e) 144 hours. sBTLA is presented on log10 scale due to non-Gaussian distribution.

https://doi.org/10.1371/journal.pone.0169176.g005

There was no difference in baseline sBTLA concentration between patients with severe sepsis and septic shock (SIRS-based sepsis definitions) (p = 0.68). sBTLA concentrations were however higher in patients with septic shock than in those with sepsis (Sepsis-3 definitions) (p<0.01) (Fig 6).

thumbnail
Download:
Fig 6. Baseline sBTLA in sepsis compared to septic shock.

Patients are defined according to Sepsis-3. Sepsis (n = 23), median 8 ng/mL, IQR 5–14 vs septic shock (n = 75), median 18 ng/mL, IQR 9–31.

https://doi.org/10.1371/journal.pone.0169176.g006

18 subjects in the sepsis cohort died within 28 days after study inclusion. sBTLA levels were significantly higher at baseline in 28 day sepsis non-survivors compared to survivors (Fig 7).

thumbnail
Download:
Fig 7. sBTLA in sepsis 28 day survivors and non-survivors.

sBTLA levels at baseline in 28 day sepsis non-survivors compared to survivors: 28 (IQR 13–41, n = 17) vs 13 (IQR 8–23, n = 81) ng/mL.

https://doi.org/10.1371/journal.pone.0169176.g007

Survival analyses, estimated with Cox regression with adjustment for age, showed a non-significant relative risk of death within the first 28 days of nearly 3 per log unit increase in baseline sBTLA concentration (HR 2.7 (0.9–8.4) p = 0.08), AUC 0.66 (95% CI 0.51–0.82), and a similar association after adjustment for comorbidities. There was, however, a significant 5-fold higher relative risk of death in patients with a high (> 21 ng/mL) compared to those with a low (<9 ng/mL) baseline plasma sBTLA, with adjustment for age and comorbidites (HR 5.1, 95% CI 1.3–19, p = 0.02), but no increased risk for patients with intermediate (10–21 ng/mL) compared to low baseline sBTLA (HR 1.2, 95% CI 0.2–6.2, p = 0.81) (Fig 8).

thumbnail
Download:
Fig 8. Survival curves for patients in the sepsis cohort at different baseline sBTLA concentrations.

Survival curves based on the 33rd and 67th percentile of baseline sBTLA concentration day 0–28 after study inclusion.

https://doi.org/10.1371/journal.pone.0169176.g008

With a cut-off at 21 ng/mL, and after adjustment for age and comorbidities, a significant association was seen between the higher level of baseline sBTLA and mortality (HR 4.6, 95% CI 1.6–12.8, p<0.01, compared to <21 ng/mL). The association remained significant after adjustment for APACHE II score (HR 4.8, 95% CI 1.7–14, p<0.01). The association between baseline log10 sBTLA and mortality disappeared when adjustment was made for log10 lactate (HR 1.3 (95% CI 0.3–4.9, p = 0.7), but if categorized as high-low, HR for sBTLA >21 ng/mL was 2.9 (95% CI 0.9–8.7), p = 0.06 after adjustment for lactate. Time-varying Cox Regression showed an even stronger and significant association between sBTLA and mortality, both when evaluated on a continuous log-10 scale and as a categorical (high vs. low) marker. Adjustment for s-lactate was not possible in these models due to missing data (Table 2).

thumbnail
Download:
Table 2. Relative risk for 28 day mortality in the sepsis cohort.

https://doi.org/10.1371/journal.pone.0169176.t002

In ICU controls 28 day mortality was 39%. There was no difference in sBTLA concentration between survivors and non-survivors among ICU controls (at baseline p = 0.5). sBTLA >21 ng/mL at baseline showed no significant age-adjusted association to mortality (HR 1.6, 95% CI 0.4–6.5, p = 0.5) and sBTLA levels did not correlate significantly to SOFA score in this cohort (at baseline rho 0.3, p = 0.1).

sPD-1 concentrations did not differ significantly between subgroups according to old (severe sepsis-septic shock) or new (sepsis-septic shock) sepsis definitions (p = 0.2 and p = 0.7 respectively) or between 28 day survivors and non-survivors in the sepsis cohort (p = 0.9) and ICU controls (p = 0.7), nor did they correlate to markers of disease severity (data not shown). There was no difference in risk of death between low versus intermediate and high levels of sPD-1 at baseline (adjusted for age) in either the sepsis (high vs low: HR 1.0, 95% CI 0.3–3.2 p = 0.98) or the ICU control cohort (data not shown).

Age and sex

In ICU controls, but not in the sepsis cohort, sBTLA was lower in subjects over 65 years of age at baseline (5 ng/mL, IQR 2–8 vs 11, IQR 9–40, p<0.01). There was no significant correlation between baseline sBTLA and age in the sepsis cohort (rho -0.1, p = 0.3). Only one HC was above 65 years of age, hence age group comparison was not possible, but there was no significant correlation between sBTLA and age in this cohort (rho -0.08, p = 0.7). PD-1 concentrations did not correlate to age (Spearman’s rho -0.09, p = 0.3). There was no statistically significant difference related to sex between study cohorts in the concentration of the studied markers (data not shown).

Discussion

In the present study we found that plasma concentrations of the soluble isoform of the co-inhibitory receptor B- and T-lymphocyte Attenuator (sBTLA) were low in healthy individuals and elevated early in critical illness, most markedly in sepsis. We found a correlation to severity of disease and, notably, a nearly 5-fold increase in mortality for sepsis patients with a baseline sBTLA concentration of >21 ng/mL compared to those with a concentration below that threshold. The association with mortality remained consistent after adjustment for comorbidities, but decreased after adjustment for baseline lactate concentration. The ability to diagnose sepsis in the critically ill cohort was however low. Evaluation of sBTLA as a dynamic marker confirmed an association to mortality. Unlike sBTLA, Programmed Death 1 (sPD-1) concentrations were lower in patients with sepsis and septic shock than in healthy control subjects, with no correlation to disease severity, and neither sPD-1 was a good biomarker to diagnose sepsis in critically ill patients. Soluble Cytotoxic T Lymphocyte-associated Antigen 4 (sCTLA-4) concentrations were low or undetectable.

This is to the best of our knowledge the first study of soluble co-inhibitory molecules in sepsis. It is not known if or how the plasma concentrations correlate to the expression of the membrane-bound correspondent receptors. Previous studies on sPD-1 have shown elevated serum and plasma levels and a correlation to disease activity in for example rheumatoid arthritis and systemic sclerosis, which is interesting in light of the observation that PD-1 deletion results in susceptibility to autoimmune disease in mice, but on the other hand sPD-1 was decreased in idiopathic thrombocytopenic purpura. sPD-1 in plasma has also been reported to be associated with development of hepatocellular carcinoma in chronic hepatitis B [15, 34–37]. Considering that PD-1 is not expressed on naïve T cells, and that there are some earlier reports of up-regulated peripheral blood T cell expression of PD-1 in septic shock, it was surprising to find plasma sPD-1 to be lower in patients with sepsis than in HC. In the early phase of sepsis there is a decrease in the absolute number of circulating lymphocytes as demonstrated for example by Boomer et al [20]. It is possible that this could explain why soluble PD-1 is decreased while PD-1 expression is increased on circulating T cells. Unfortunately we had incomplete data on lymphocyte count and could therefore not evaluate if sPD-1 levels correlated to lymphocyte levels. It has been demonstrated that sPD-1 can block the inhibitory effect of membrane-bound PD-1 on T-cell activation in vitro [38]. The observed decrease of plasma soluble PD-1 in the sepsis cohort could be explained by sPD-1 consumption upon immune activation as a result of ligation to PD-L1 and–L2, resulting in a loss of T cell inhibition in the acute phase of infection. It is also possible that a pro-inflammatory stimulus causes a switch in PD-1 gene transcription in favor of the membrane-bound isoform. Interfering with PD-1-PD-L1/PD-L2 interaction has been proposed as a method of counteracting immunosuppression in sepsis, and animal models have shown positive results in terms of immune cell function and survival [22, 23]. We found no evidence for an upregulation of sCTLA-4 in sepsis. Previous studies on soluble CTLA-4 show increased plasma concentrations in several autoimmune disorders, but low or undetectable levels in healthy individuals [39–42]. There might be certain pathogenic mechanisms present in autoimmune disease, but not in sepsis, resulting in an increased production of sCTLA-4. Perhaps due to the limited sensitivity of commercially available ELISA kits, sCTLA-4 is difficult to study.

Unlike for sPD-1 and sCTLA-4 there are no publications on sBTLA in other disease states. The existence of sBTLA in sera from healthy individuals has been demonstrated by Wang et al [43]. In their study, but not in ours, sBTLA was positively correlated with age. Studies on cell expression of BTLA in sepsis show conflicting results. Shubin and co-workers found the percentage of peripheral blood CD4+ T cells expressing BTLA to be higher in patients with sepsis than in patients with non-infectious SIRS, while Boomer et al recorded no difference between patients sampled within 24 hours after sepsis onset and healthy controls (HC) [20, 21]. In the Shubin study, patients were however sampled later in the course of sepsis which makes comparison difficult. In the latter study, T cell expression of BTLA was only 50% in HC while Boomer et al found it to be >90% in both septic patients and HC. A recent study by Shao et al, where patients were sampled within 24 hours after Emergency Department admission, showed 90% BTLA+ CD4+ T cells in healthy control subjects, with a lower expression in patients with sepsis, even lower in severe sepsis and lowest in septic shock, [44]. Besides higher expression of BTLA in sepsis, Shubin and co-workers reported a higher proportion of CD4+ T cells expressing BTLA in patients who subsequently developed a nosocomial infection [21]. Although many of the ICU control subjects in our study developed an infection during their ICU stay, we did not analyze this further due to the small number of subjects in this cohort.

The BTLA-HVEM pathway is complicated by the fact that HVEM has multiple partner molecules through which cross-linking results in either a co-stimulatory (LIGHT and LTα) or a co-inhibitory (BTLA and CD160) signal [14]. Therefore BTLA might be a more fine-tuned immune regulator than CTLA-4 and PD-1. The origin of sBTLA remains to be elucidated. Perhaps it is generated through alternative splicing of mRNA as has been demonstrated for sPD-1 and sCTLA-4 [26, 27, 45]. Another potential source of the measured BTLA is proteolytic cleavage of the outer part of the membrane-bound receptor, possibly representing a means of controlling T cell inhibition. Thirdly, although specific antibodies to detect sBTLA and sPD-1 have been developed, at the time of carrying out these experiments there were no commercially available ELISA kits able to discriminate soluble BTLA and PD-1 from the membrane bound isoform, and we cannot exclude that some BTLA and PD-1 detected in the analyses may be connected to cell fragments in plasma [46]. In this study, baseline plasma sBTLA concentrations were highest in the sepsis cohort suggesting that sBTLA is a marker for an activated pathway in the immune system, triggered by infection. sBTLA concentrations and disease severity in the sepsis cohort were associated; sBTLA and SOFA score were correlated and septic shock patients had higher concentrations than those with sepsis without shock. Moreover, we found an association between sBTLA and 28 day mortality for patients in the sepsis cohort, when assessed by baseline value, and, with reservation for the possible bias of the imputation method used for missing values, on an even stronger level when evaluated as a dynamic marker, indicating that repeated measurements of sBTLA during the course of sepsis may be valuable. In keeping with our findings, it was shown in an experimental sepsis model that BTLA-deficient mice were protected from kidney injury and had a higher rate of survival [47]. However, in a recently published study by the same group, anti-BTLA antibody administered intraperitoneally directly after cecal ligation and puncture in mice showed opposite effects [48]. Taken together, our results suggest that, although soluble co-inhibitory receptors do not appear to be good markers to diagnose sepsis in the critically ill patient, sBTLA may be of interest as a prognostic indicator, perhaps in combination with lactate and other biomarkers, and that further studies on the immune pathways involving sBTLA may be of value in clarifying the pathogenesis in human sepsis.

One factor of uncertainty, in this and other sepsis studies, is that the course of sepsis differs between patients and infecting agents. The time point of onset of infection can also be difficult to determine, which was the case in 12/101 sepsis patients in this study. In patients where it could be defined, the delay from the first symptoms of an infection and drawing of the first blood sample was in median 2 days (IQR 1–3, range 0–19). This study has other limitations. There is a substantial shortfall in plasma samples in the latter half of the study period, due to missing samples or patients being transferred from the ICU because of recovery or for other reasons, resulting in changed composition of the study group as well as a significantly reduced number of ICU controls, adding some difficulty to drawing conclusions on the dynamics of the studied markers. Furthermore almost half of the ICU controls developed an infection during the study period. To reduce bias due to these factors, we limited group comparison of the sepsis and ICU control cohorts and sepsis survivors and non-survivors to the baseline measurement. Considering the relatively few study subjects and the fact that there are no previous publications on this marker in sepsis patients, the chosen threshold sBTLA concentration for increased risk of mortality may not be optimal and alternative threshold values should be tested in future studies.

Finally, this is an observational study and no conclusions can be drawn in terms of the underlying immunological mechanisms. Future mechanistic studies are required to examine the biological activity of soluble BTLA and why high levels are associated with mortality. This may also clarify whether receptor intervention could be favorable in human sepsis or if sBTLA will serve better as a biomarker.

Our results suggest that sBTLA may be valuable as a dynamic prognostic marker in sepsis. However, additional studies, including patients with less severe disease, and ideally with correlation to known markers of sepsis-induced immunosuppression, are needed to evaluate the biological significance of sBTLA and whether it may be useful, possibly in conjunction with other biomarkers for risk stratification in the septic patient.

Supporting Information

S1 File. Data.

Data file containing all raw data used in the statistical analyses.

https://doi.org/10.1371/journal.pone.0169176.s001

(XLSX)

Acknowledgments

We wish to thank research nurse Gunilla Herman at Karolinska University Hospital for invaluable help in collecting and registering clinical data, study patients and controls for their participation and Ulla Larsson, biomedical technician, for excellent laboratory assistance. The study was conducted with the financial support from research grants from the Research Committee Region Örebro County, Nyckelfonden Region Örebro County, the Olle Engkvist fund, the Signe and Olof Wallenius trust, ALF project funding, and the Karolinska University Hospital, Huddinge Research Foundation.

Author Contributions

  1. Conceptualization: AL OH JSC.
  2. Data curation: AL OH JSC AM.
  3. Formal analysis: AL AM OH JSC.
  4. Funding acquisition: AL OH JSC.
  5. Investigation: AL JSC.
  6. Methodology: AL OH JSC AM.
  7. Project administration: OH.
  8. Resources: OH JSC AM.
  9. Software: AM.
  10. Supervision: OH JSC.
  11. Validation: AL OH JSC AM.
  12. Visualization: AL OH JSC AM.
  13. Writing – original draft: AL.
  14. Writing – review & editing: AL OH JSC AM.

References

  1. 1. Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012. JAMA. 2014;311(13):1308–16. Epub 2014/03/19. pmid:24638143
  2. 2. Stevenson EK, Rubenstein AR, Radin GT, Wiener RS, Walkey AJ. Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis*. Crit Care Med. 2014;42(3):625–31. Epub 2013/11/10. PubMed Central PMCID: PMCPmc4313930. pmid:24201173
  3. 3. Volk HD, Reinke P, Krausch D, Zuckermann H, Asadullah K, Muller JM, et al. Monocyte deactivation—rationale for a new therapeutic strategy in sepsis. Intensive Care Med. 1996;22 Suppl 4:S474–81. Epub 1996/10/01.
  4. 4. Boomer JS, To K, Chang KC, Takasu O, Osborne DF, Walton AH, et al. Immunosuppression in patients who die of sepsis and multiple organ failure. JAMA. 2011;306(23):2594–605. Epub 2011/12/22. PubMed Central PMCID: PMCPmc3361243. pmid:22187279
  5. 5. Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–74. Epub 2013/11/16. PubMed Central PMCID: PMCPmc4077177. pmid:24232462
  6. 6. Hamers L, Kox M, Pickkers P. Sepsis-induced immunoparalysis: mechanisms, markers, and treatment options. Minerva Anestesiol. 2015;81(4):426–39. Epub 2014/06/01. pmid:24878876
  7. 7. Gogos CA, Drosou E, Bassaris HP, Skoutelis A. Pro- versus anti-inflammatory cytokine profile in patients with severe sepsis: a marker for prognosis and future therapeutic options. J Infect Dis. 2000;181(1):176–80. Epub 1999/12/23. pmid:10608764
  8. 8. Schulte W, Bernhagen J, Bucala R. Cytokines in sepsis: potent immunoregulators and potential therapeutic targets—an updated view. Mediators Inflamm. 2013;2013:165974. Epub 2013/07/16. PubMed Central PMCID: PMCPmc3703895. pmid:23853427
  9. 9. Roger PM, Hyvernat H, Ticchioni M, Kumar G, Dellamonica J, Bernardin G. The early phase of human sepsis is characterized by a combination of apoptosis and proliferation of T cells. J Crit Care. 2012;27(4):384–93. Epub 2012/07/25. pmid:22824083
  10. 10. Bernard A, Lamy, Alberti I. The two-signal model of T-cell activation after 30 years. Transplantation. 2002;73(1 Suppl):S31–5. Epub 2002/01/26. pmid:11810059
  11. 11. Boulougouris G, McLeod JD, Patel YI, Ellwood CN, Walker LS, Sansom DM. Positive and negative regulation of human T cell activation mediated by the CTLA-4/CD28 ligand CD80. J Immunol. 1998;161(8):3919–24. Epub 1998/10/21. pmid:9780158
  12. 12. Freeman GJ, Long AJ, Iwai Y, Bourque K, Chernova T, Nishimura H, et al. Engagement of the PD-1 immunoinhibitory receptor by a novel B7 family member leads to negative regulation of lymphocyte activation. J Exp Med. 2000;192(7):1027–34. Epub 2000/10/04. PubMed Central PMCID: PMCPmc2193311. pmid:11015443
  13. 13. Wang XF, Chen YJ, Wang Q, Ge Y, Dai Q, Yang KF, et al. Distinct expression and inhibitory function of B and T lymphocyte attenuator on human T cells. Tissue Antigens. 2007;69(2):145–53. Epub 2007/01/30. pmid:17257317
  14. 14. Cai G, Freeman GJ. The CD160, BTLA, LIGHT/HVEM pathway: a bidirectional switch regulating T-cell activation. Immunol Rev. 2009;229(1):244–58. Epub 2009/05/12. pmid:19426226
  15. 15. Nishimura H, Nose M, Hiai H, Minato N, Honjo T. Development of lupus-like autoimmune diseases by disruption of the PD-1 gene encoding an ITIM motif-carrying immunoreceptor. Immunity. 1999;11(2):141–51. Epub 1999/09/15. pmid:10485649
  16. 16. Oya Y, Watanabe N, Owada T, Oki M, Hirose K, Suto A, et al. Development of autoimmune hepatitis-like disease and production of autoantibodies to nuclear antigens in mice lacking B and T lymphocyte attenuator. Arthritis Rheum. 2008;58(8):2498–510. Epub 2008/08/01. PubMed Central PMCID: PMCPmc2782777. pmid:18668554
  17. 17. Waterhouse P, Penninger JM, Timms E, Wakeham A, Shahinian A, Lee KP, et al. Lymphoproliferative disorders with early lethality in mice deficient in Ctla-4. Science. 1995;270(5238):985–8. Epub 1995/11/10. pmid:7481803
  18. 18. Guignant C, Lepape A, Huang X, Kherouf H, Denis L, Poitevin F, et al. Programmed death-1 levels correlate with increased mortality, nosocomial infection and immune dysfunctions in septic shock patients. Crit Care. 2011;15(2):R99. Epub 2011/03/23. PubMed Central PMCID: PMCPmc3219369. pmid:21418617
  19. 19. Zhang Y, Li J, Lou J, Zhou Y, Bo L, Zhu J, et al. Upregulation of programmed death-1 on T cells and programmed death ligand-1 on monocytes in septic shock patients. Crit Care. 2011;15(1):R70. Epub 2011/02/26. PubMed Central PMCID: PMCPmc3222003. pmid:21349174
  20. 20. Boomer JS, Shuherk-Shaffer J, Hotchkiss RS, Green JM. A prospective analysis of lymphocyte phenotype and function over the course of acute sepsis. Crit Care. 2012;16(3):R112. Epub 2012/06/30. PubMed Central PMCID: PMCPmc3580670. pmid:22742734
  21. 21. Shubin NJ, Monaghan SF, Heffernan DS, Chung CS, Ayala A. B and T lymphocyte attenuator expression on CD4+ T-cells associates with sepsis and subsequent infections in ICU patients. Crit Care. 2013;17(6):R276. Epub 2013/12/03. PubMed Central PMCID: PMCPmc4057112. pmid:24289156
  22. 22. Brahmamdam P, Inoue S, Unsinger J, Chang KC, McDunn JE, Hotchkiss RS. Delayed administration of anti-PD-1 antibody reverses immune dysfunction and improves survival during sepsis. J Leukoc Biol. 2010;88(2):233–40. Epub 2010/05/21. pmid:20483923
  23. 23. Zhang Y, Zhou Y, Lou J, Li J, Bo L, Zhu K, et al. PD-L1 blockade improves survival in experimental sepsis by inhibiting lymphocyte apoptosis and reversing monocyte dysfunction. Crit Care. 2010;14(6):R220. Epub 2010/12/02. PubMed Central PMCID: PMCPmc3220038. pmid:21118528
  24. 24. Chang KC, Burnham CA, Compton SM, Rasche DP, Mazuski RJ, McDonough JS, et al. Blockade of the negative co-stimulatory molecules PD-1 and CTLA-4 improves survival in primary and secondary fungal sepsis. Crit Care. 2013;17(3):R85. Epub 2013/05/15. PubMed Central PMCID: PMCPmc3706819. pmid:23663657
  25. 25. Inoue S, Bo L, Bian J, Unsinger J, Chang K, Hotchkiss RS. Dose-dependent effect of anti-CTLA-4 on survival in sepsis. Shock. 2011;36(1):38–44. Epub 2011/03/04. PubMed Central PMCID: PMCPmc3412306. pmid:21368717
  26. 26. Oaks MK, Hallett KM, Penwell RT, Stauber EC, Warren SJ, Tector AJ. A native soluble form of CTLA-4. Cell Immunol. 2000;201(2):144–53. Epub 2000/06/01. pmid:10831323
  27. 27. Nielsen C, Ohm-Laursen L, Barington T, Husby S, Lillevang ST. Alternative splice variants of the human PD-1 gene. Cell Immunol. 2005;235(2):109–16. Epub 2005/09/21. pmid:16171790
  28. 28. Wan B, Nie H, Liu A, Feng G, He D, Xu R, et al. Aberrant regulation of synovial T cell activation by soluble costimulatory molecules in rheumatoid arthritis. J Immunol. 2006;177(12):8844–50. Epub 2006/12/05. pmid:17142787
  29. 29. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–55. Epub 1992/06/01. pmid:1303622
  30. 30. Shankar-Hari M, Phillips GS, Levy ML, Seymour CW, Liu VX, Deutschman CS, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):775–87. Epub 2016/02/24. pmid:26903336
  31. 31. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics. 1979:65–70.
  32. 32. Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581–92.
  33. 33. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29. Epub 1985/10/01. pmid:3928249
  34. 34. Greisen SR, Rasmussen TK, Stengaard-Pedersen K, Hetland ML, Horslev-Petersen K, Hvid M, et al. Increased soluble programmed death-1 (sPD-1) is associated with disease activity and radiographic progression in early rheumatoid arthritis. Scand J Rheumatol. 2014;43(2):101–8. Epub 2013/11/05. pmid:24182347
  35. 35. Atesoglu EB, Tarkun P, Demirsoy ET, Geduk A, Mehtap O, Batman A, et al. Soluble Programmed Death 1 (PD-1) Is Decreased in Patients With Immune Thrombocytopenia (ITP): Potential Involvement of PD-1 Pathway in ITP Immunopathogenesis. Clin Appl Thromb Hemost. 2014. Epub 2014/12/17.
  36. 36. Yanaba K, Hayashi M, Yoshihara Y, Nakagawa H. Serum levels of soluble programmed death-1 and programmed death ligand-1 in systemic sclerosis: Association with extent of skin sclerosis. J Dermatol. 2016. Epub 2016/03/08.
  37. 37. Cheng HY, Kang PJ, Chuang YH, Wang YH, Jan MC, Wu CF, et al. Circulating programmed death-1 as a marker for sustained high hepatitis B viral load and risk of hepatocellular carcinoma. PLoS One. 2014;9(11):e95870. Epub 2014/11/27. PubMed Central PMCID: PMCPmc4245192. pmid:25427199
  38. 38. Liu C, Jiang J, Gao L, Wang X, Hu X, Wu M, et al. Soluble PD-1 aggravates progression of collagen-induced arthritis through Th1 and Th17 pathways. Arthritis Res Ther. 2015;17:340. Epub 2015/11/27. PubMed Central PMCID: PMCPmc4659197. pmid:26608464
  39. 39. Saverino D, Pesce G, Antola P, Porcelli B, Brusca I, Villalta D, et al. High levels of soluble CTLA-4 are present in anti-mitochondrial antibody positive, but not in antibody negative patients with primary biliary cirrhosis. PLoS One. 2014;9(11):e112509. Epub 2014/11/11. PubMed Central PMCID: PMCPmc4226553. pmid:25383768
  40. 40. Simone R, Pesce G, Antola P, Rumbullaku M, Bagnasco M, Bizzaro N, et al. The soluble form of CTLA-4 from serum of patients with autoimmune diseases regulates T-cell responses. 2014;2014:215763. pmid:24605322
  41. 41. Saverino D, Brizzolara R, Simone R, Chiappori A, Milintenda-Floriani F, Pesce G, et al. Soluble CTLA-4 in autoimmune thyroid diseases: relationship with clinical status and possible role in the immune response dysregulation. Clin Immunol. 2007;123(2):190–8. Epub 2007/02/27. pmid:17321799
  42. 42. Simone R, Brizzolara R, Chiappori A, Milintenda-Floriani F, Natale C, Greco L, et al. A functional soluble form of CTLA-4 is present in the serum of celiac patients and correlates with mucosal injury. Int Immunol. 2009;21(9):1037–45. Epub 2009/07/25. pmid:19625381
  43. 43. Wang X, Zhu G Fau—Huang Z, Huang Z Fau—Cao L, Cao L Fau—Chen Y, Chen Y Fau—Wang Q, Wang Q Fau—Zhang X, et al. Application of monoclonal antibodies in a sandwich enzyme-linked immunosorbent assay for identification and detection of soluble human B and T lymphocyte attenuator. (1557–8348 (Electronic)).
    • 44. Shao R, Li CS, Fang Y, Zhao L, Hang C. Low B and T lymphocyte attenuator expression on CD4(+) T cells in the early stage of sepsis is associated with the severity and mortality of septic patients: a prospective cohort study. Crit Care. 2015;19(1):308. Epub 2015/09/04. PubMed Central PMCID: PMCPmc4556404.
    • 45. Magistrelli G, Jeannin P, Herbault N, Benoit De Coignac A, Gauchat JF, Bonnefoy JY, et al. A soluble form of CTLA-4 generated by alternative splicing is expressed by nonstimulated human T cells. Eur J Immunol. 1999;29(11):3596–602. Epub 1999/11/11. pmid:10556814
    • 46. Wang X, Zhu G, Huang Z, Cao L, Chen Y, Wang Q, et al. Application of monoclonal antibodies in a sandwich enzyme-linked immunosorbent assay for identification and detection of soluble human B and T lymphocyte attenuator. Hybridoma (Larchmt). 2012;31(6):417–23. Epub 2012/12/19.
    • 47. Shubin NJ, Chung CS, Heffernan DS, Irwin LR, Monaghan SF, Ayala A. BTLA expression contributes to septic morbidity and mortality by inducing innate inflammatory cell dysfunction. J Leukoc Biol. 2012;92(3):593–603. Epub 2012/03/31. PubMed Central PMCID: PMCPmc3427605. pmid:22459947
    • 48. Cheng T, Bai J, Chung CS, Chen Y, Biron BM, Ayala A. Enhanced Innate Inflammation Induced by Anti-BTLA Antibody in Dual Insult Model of Hemorrhagic Shock/Sepsis. Shock. 2016;45(1):40–9. Epub 2015/12/18. PubMed Central PMCID: PMCPmc4683612. pmid:26674453