Predicting ICU Mortality: the PhysioNet/Computing in Cardiology Challenge 2012

The development of methods for prediction of mortality rates in Intensive Care Unit (ICU) populations has been motivated primarily by the need to compare the efficacy of medications, care guidelines, surgery, and other interventions when, as is common, it is necessary to control for differences in severity of illness or trauma, age, and other factors. For example, comparing overall mortality rates between trauma units in a community hospital, a teaching hospital, and a military field hospital is likely to reflect the differences in the patient populations more than any differences in standards of care. Acuity scores such as APACHE and SAPS-II are widely used to account for these differences in the context of such studies.

By contrast, the focus of the PhysioNet/CinC Challenge 2012 is to develop methods for patient-specific prediction of in-hospital mortality. Participants will use information collected during the first two days of an ICU stay to predict which patients survive their hospitalizations, and which patients do not.

Data for the Challenge

The data used for the challenge consist of records from 12,000 ICU stays. All patients were adults who were admitted for a wide variety of reasons to cardiac, medical, surgical, and trauma ICUs. ICU stays of less than 48 hours have been excluded.

Four thousand records comprise training set A, and the remaining records form test sets B and C. Outcomes are provided for the training set records, and are withheld for the test set records.

Up to 41 variables were recorded at least once during the first 48 hours after admission to the ICU. Not all variables are available in all cases, however. Five of these variables are general descriptors (collected on admission), and the remainder are time series, for which multiple observations may be available.

Each observation has an associated time-stamp indicating the elapsed time of the observation since ICU admission in each case, in hours and minutes. Thus, for example, a time stamp of 35:19 means that the associated observation was made 35 hours and 19 minutes after the patient was admitted to the ICU.

Each record is stored as a comma-separated value (CSV) text file. To simplify downloading, participants may download a zip file or tarball containing all of training set A or test set B. Test set C will be used for validation only and will not be made available to participants.

Five additional outcome-related descriptors, described below, are known for each record. These are stored in separate CSV text files for each of sets A, B, and C, but only those for set A are available to challenge participants.

All valid values for general descriptors, time series variables, and outcome-related descriptors are non-negative (≥ 0). A value of -1 indicates missing or unknown data (for example, if a patient's height was not recorded).

General descriptors

As noted, these five descriptors are collected at the time the patient is admitted to the ICU. Their associated time-stamps are set to 00:00 (thus they appear at the beginning of each patient's record).

RecordID (a unique integer for each ICU stay) Age (years) Gender (0: female, or 1: male) Height (cm) Weight (kg)*.

Time Series

These 37 variables may be observed once, more than once, or not at all in some cases:

Albumin (g/dL) ALP [Alkaline phosphatase (IU/L)] ALT [Alanine transaminase (IU/L)] AST [Aspartate transaminase (IU/L)] Bilirubin (mg/dL) BUN [Blood urea nitrogen (mg/dL)] Cholesterol (mg/dL) Creatinine [Serum creatinine (mg/dL)] DiasABP [Invasive diastolic arterial blood pressure (mmHg)] FiO2 [Fractional inspired O2 (0-1)] GCS [Glasgow Coma Score (3-15)] Glucose [Serum glucose (mg/dL)] HCO3 [Serum bicarbonate (mmol/L)]

HCT [Hematocrit (%)] HR [Heart rate (bpm)] K [Serum potassium (mEq/L)] Lactate (mmol/L) Mg [Serum magnesium (mmol/L)] MAP [Invasive mean arterial blood pressure (mmHg)] MechVent [Mechanical ventilation respiration (0:false, or 1:true)] Na [Serum sodium (mEq/L)] NIDiasABP [Non-invasive diastolic arterial blood pressure (mmHg)] NIMAP [Non-invasive mean arterial blood pressure (mmHg)] NISysABP [Non-invasive systolic arterial blood pressure (mmHg)]

PaCO2 [partial pressure of arterial CO2 (mmHg)] PaO2 [Partial pressure of arterial O2 (mmHg)] pH [Arterial pH (0-14)] Platelets (cells/nL) RespRate [Respiration rate (bpm)] SaO2 [O2 saturation in hemoglobin (%)] SysABP [Invasive systolic arterial blood pressure (mmHg)] Temp [Temperature (°C)] TropI [Troponin-I (μg/L)] TropT [Troponin-T (μg/L)] Urine [Urine output (mL)] WBC [White blood cell count (cells/nL)] Weight (kg)*

The time series measurements are recorded in chronological order within each record, and the associated time stamps indicate the elapsed time since admission to the ICU. Measurements may be recorded at regular intervals ranging from hourly to daily, or at irregular intervals as required. Not all time series are available in all cases.

In a few cases, such as blood pressure, different measurements made using two or more methods or sensors may be recorded with the same or only slightly different time-stamps. Occasional outliers should be expected as well.

 ^* Note that Weight is both a general descriptor (recorded on admission) and a time series variable (often measured hourly, for estimating fluid balance).

Outcome-related Descriptors

The outcome-related descriptors are kept in a separate CSV text file for each of the three record sets; as noted, only the file associated with training set A is available to participants. Each line of the outcomes file contains these descriptors:

• RecordID (defined as above)
• SAPS-I score (Le Gall et al., 1984)
• SOFA score (Ferreira et al., 2001)
• Length of stay (days)
• Survival (days)
• In-hospital death (0: survivor, or 1: died in-hospital)

The Length of stay is the number of days between the patient's admission to the ICU and the end of hospitalization (including any time spent in the hospital after discharge from the ICU). If the patient's death was recorded (in or out of hospital), then Survival is the number of days between ICU admission and death; otherwise, Survival is assigned the value -1. Since patients who spent less than 48 hours in the ICU have been excluded, Length of stay and Survival never have the values 0 or 1 in the challenge data sets. Given these definitions and constraints,

Survival > Length of stay  ⇒  Survivor
Survival = -1  ⇒  Survivor
2 ≤ Survival ≤ Length of stay  ⇒  In-hospital death

Entering the Challenge

The rules for participating in the Challenge will be posted here shortly.

To begin, we recommend studying the training set as preparation for the Challenge itself. In particular, note that the SAPS-I score can be calculated readily from the time series, as the sample entry does. To succeed in the Challenge, you should aim to outperform the sample entry.

Awards will be presented to the most successful eligible participants during Computing in Cardiology 2012. To be eligible for an award, you must:

1. Submit a preliminary Challenge entry via PhysioNetWorks no later than 25 April 2012.
2. Submit an acceptable abstract on your work on the Challenge to CinC no later than 1 May 2012.
3. Submit a final Challenge entry via PhysioNetWorks no later than 25 August 2012.
4. Submit a full (4-page) paper on your work on the Challenge to CinC no later than 1 September 2012.
5. Attend CinC 2012 (9-12 September 2012, in Krakow, Poland) and present your work there.

Challenge scoring

As in previous challenges, participants may compete in multiple events:

Event 1

The goal is to predict in-hospital mortality with the greatest accuracy.

Event 2

The goal is to predict in-hospital survival with the greatest accuracy.

Scoring for these events is based on four metrics: sensitivity (Se), specificity (Sp), positive predictivity (⁺P), and negative predictivity (^-P). We define the numbers of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) as below:

Outcome		Observed
		Death	Survivor
Predicted	Death	TP	FP
	Survivor	FN	TN

Using these definitions, the four metrics are given by:

Se = TP / (TP + FN)	[the fraction of in-hospital deaths that are predicted]
+P = TP / (TP + FP)	[the fraction of correct predictions of in-hospital deaths]
Sp = TN / (TN + FP)	[the fraction of survivors that are predicted]
-P = TN / (TN + FN)	[the fraction of correct predictions of survivors]

Suggested readings on acuity scores and mortality prediction

APACHE (1981)

Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE - acute physiology and chronic health evaluation: A physiologically based classification system. Critical Care Medicine 9(8):591-597 (1981 Aug).

SAPS-I (1984)

Le Gall JR, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, Mercier P, Thomas R, Villers D. A simplified acute physiology score for ICU patients. Critical Care Medicine 12(11):975-977 (1984 Nov).

MPM-I (1985)

Lemeshow S, Teres D, Pastides H, Avrunin JS, Steingrub JS. A method for predicting survival and mortality of ICU patients using objectively derived weights. Critical Care Medicine 13(7):519-525 (1985 Jul).

APACHE II (1985)

Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Critical Care Medicine 13(10):818-29 (1985 Oct).

APACHE III (1991)

Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100(6):1619-1636 (1991 Dec).

MPM II (1993)

Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J. Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270(20):2478-86 (1993 Nov 24).

SAPS-II (1993)

Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA 270(24):2957-2963 (1993 Dec 22-29).

Elixhauser Comorbidity Index (1998)

Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 36(1):8-27 (1998 Jan).

SOFA (2001)

Ferreira, FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 286(14):1754-1758 (2001 Oct 10).

SAPS 3 (2005)

Metnitz PGH, Moreno RP, Almeida E, Jordan B, Bauer P, Abizanda Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR. SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med 31:1336-1344 (2005).
Moreno RP, Metnitz PGH, Almeida E, Jordan B, Bauer P, Abizanda Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR. SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345-1355 (2005).

[Event-based method] (2006)

Silva A, Cortez P, Santos MF, Gomes L, Neves J. Mortality assessment in intensive care units via adverse events using artificial neural networks. Artificial Intelligence in Medicine 36(3):223-234 (2006 Mar).

APACHE IV (2006)

Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Critical Care Medicine 34(5):1297-310 (2006 May).

MPM0-III (2007)

Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AA. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Critical Care Medicine 35(3):827-835 (2007 Mar).

[Real-time method] (2009)

Hug CW, Szolovits P. ICU Acuity: Real-time models versus daily models. AMIA Annual Symposium Proceedings, 260-264 (2009).