5.4 Cumulative hazards and life tables in software

Software 5.4.1 (Nelson–Aalen by hand). Example 5.1.2: the cumulative hazard and its standard error at each event time, the survival estimate \(\exp (-\hat H)\), and the Kaplan–Meier estimate for comparison:

> library(survival)
> tm <- c(3, 5, 6, 8, 10, 12, 15, 18, 22, 24)
> ev <- c(1, 0, 1, 1, 0, 1, 0, 1, 0, 0)
> km <- survfit(Surv(tm, ev) ~ 1)
> k <- km$n.event > 0                     # rows at event times
> H <- cumsum(km$n.event[k] / km$n.risk[k])   # Nelson-Aalen
> v <- cumsum(km$n.event[k] / km$n.risk[k]^2) # its variance
> round(cbind(t = km$time[k], H_NA = H, se = sqrt(v), exp_minus_H = exp(-H),
+             KM = km$surv[k]), 4)
      t   H_NA     se exp_minus_H     KM
[1,]  3 0.1000 0.1000      0.9048 0.9000
[2,]  6 0.2250 0.1601      0.7985 0.7875
[3,]  8 0.3679 0.2146      0.6922 0.6750
[4,] 12 0.5679 0.2933      0.5667 0.5400
[5,] 18 0.9012 0.4440      0.4061 0.3600

Reading the output. \(\hat H\) rises by \(d_j/n_j\) at each death, reaching \(0.901\) at \(18\) months with standard error \(0.444\). Every value of \(\exp (-\hat H)\) exceeds the Kaplan–Meier value, as Proposition 5.1.1 requires, by up to \(0.046\) at the last death.

Python: lifelines.NelsonAalenFitter().fit(tm, ev).cumulative_hazard_.

Software 5.4.2 (A Weibull check). The Nelson–Aalen estimate for lung, a least-squares line of \(\log \hat H\) on \(\log t\) through the death times, and a comparison of \(\hat H(365)\) with \(-\log \hat S_{\text {KM}}(365)\):

> fit <- survfit(Surv(time, status) ~ 1, data = lung)
> ok <- fit$n.event > 0 & fit$surv > 0
> Hl <- cumsum(fit$n.event / fit$n.risk)[ok]
> x <- log(fit$time[ok]); y <- log(Hl)
> round(coef(lm(y ~ x)), 4)               # slope estimates the Weibull shape
(Intercept)           x
    -7.3392      1.2096
> round(c(H_at_365 = Hl[max(which(fit$time[ok] <= 365))],
+         minus_log_KM = -log(summary(fit, times = 365)$surv)), 4)
    H_at_365 minus_log_KM
      0.8883       0.8934

Reading the output. The slope \(1.21\) confirms an increasing hazard, as the Weibull fit found (\(1.32\)). At one year \(\hat H = 0.888\) and \(-\log \hat S_{\text {KM}} = 0.893\): with \(65\) or more at risk throughout the first year, the two estimators nearly agree.

Python: numpy.polyfit(numpy.log(t), numpy.log(H), 1).

Software 5.4.3 (An actuarial life table). Example 5.3.1: deaths and censorings by quarter-year, the numbers entering each interval, the conditional probabilities of death with the half-interval correction, and the survival at the end of each interval; then the Kaplan–Meier estimates at \(91\), \(182\), \(364\) and \(728\) days:

> br <- seq(0, 1092, by = 91)             # quarter-year intervals
> grp <- cut(lung$time, br, right = FALSE)
> d <- unname(tapply(lung$status == 2, grp, sum))       # deaths in each
> cen <- unname(tapply(lung$status == 1, grp, sum))     # and censored
> n <- nrow(lung) - c(0, cumsum(d + cen)[-length(d)])   # entering each
> nprime <- n - cen / 2                   # effective number at risk
> q <- d / nprime; S <- cumprod(1 - q)
> tab <- data.frame(start = head(br, -1), n, deaths = d, censored = cen,
+                   q = round(q, 4), S_end = round(S, 4))
> tab[1:8, ]
  start   n deaths censored      q  S_end
1     0 228     27        0 0.1184 0.8816
2    91 201     38        6 0.1919 0.7124
3   182 157     28       23 0.1924 0.5753
4   273 106     27       12 0.2700 0.4200
5   364  67     13        7 0.2047 0.3340
6   455  47     10        5 0.2247 0.2589
7   546  32      7        3 0.2295 0.1995
8   637  22      8        0 0.3636 0.1270
> round(summary(fit, times = c(91, 182, 364, 728))$surv, 4)   # KM, to compare
[1] 0.8816 0.7081 0.4092 0.1157

Reading the output. The quarterly probability of death rises from \(0.12\) in the first quarter to about \(0.19\)–\(0.27\) thereafter, the increasing hazard again. The life table’s survival at the ends of the intervals, \(0.882\), \(0.712\), …, \(0.420\) at one year and \(0.127\) at two, tracks the Kaplan–Meier values \(0.882\), \(0.708\), \(0.409\), \(0.116\) within a little over a percentage point.

Python: lifelines.utils.survival_table_from_events gives the counts, from which the table follows.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.