Skip to main content

Table 1 Agreement statistics for the best ICD code-based algorithm(s) for acute heart failure studies

From: Accuracy of heart failure ascertainment using routinely collected healthcare data: a systematic review and meta-analysis

First author

Year

Country

GS HF events

RCD source

GS source

ICD version

ICD algorithm/code pos.

Sensitivity (95% CI)

Specificity (95% CI)

PPV (95% CI)

NPV (95% CI)

Kappa (95% CI)

Huang [33]

2017

USA

369

HDD

MR

9

Best alg.

96.6 (96.3–96.8)

–

92.3 (92.0–92.6)

–

–

Rosamond [56]

2012

USA

425

HDD

MR

9

Any

95.0

17.0

62.0

–

–

1ry

43.0

95.0

93.0

–

–

Fisher [28]

1992

USA

788

ACD

MR (re-coded to ICD–9)

8

Any

89.0 (86.6–91.1)

95.4 (94.9–95.9)

71.0 (68.1–73.8)

98.6 (98.2–98.8)

–

402

1ry

85.0 (81.2–88.4)

99.2 (99.0–99.4)

87.0 (83.3–90.2)

99.1 (98.8–99.2)

–

Birman-Deych [78]

2005

USA

–

Registry

MR

9

Any

83.0

86.0

85.0

–

–

Schellenbaum [47]

2006

USA

712

HDD

MR

9

Any

80.8 (77.7–83.6)

90.4 (89.6–91.2)

53.6 (50.6–56.7)

97.2 (96.6–97.6)

–

Heckbert [32]

2004

USA

795

HDD

MR

9

Any

79.0 (76.0–81.8)

97.7 (97.6–97.9)

45.3 (42.7–48.0)

99.5 (99.4–99.6)

0.56 (0.53–0.59)

Alqaisi [23]

2009

USA

260

ACD

MR

9

Any/Der.

76.3

74.5

–

–

–

Any/Val.

78.5

68.6

–

–

–

Goff [31]

2000

USA

1376

Registry

MR

9

Any/alg.3

67.1 (64.5–69.6)

92.6 (91.7–93.4)

77.1 (74.6–79.5)

88.3 (87.3–89.3)

–

Psaty [44]

2016

USA

1863

ACD

MR

9

Any

53.5 (51.2–55.8)

89.9 (88.9–90.9)

72.5 (70.1–74.9)

79.6 (78.3–80.8)

–

1ry

27.2 (25.2–29.3)

99.0 (98.6–99.3)

93.2 (90.7–95.2)

73.2 (72.0–74.5)

–

Cohen [50]

2020

USA

1172

ACD

MR

9, 10

Any

45.6 (42.7–48.5)

88.2 (85.8–90.3)

84.0 (80.9, 86.7)

54.4 (51.7–57.0)

–

Presley [55]

2018

USA

360

HDD

EMR

9

1ry

45.1 (25.1–65.1)

99.4 (99.2–99.6)

89.7 (86.8–92.7)

93.9 (89.1–98.6)

–

Allen [77]

2014

USA

46

Registry

MR

9

Alg. 2

41.7 (16.1–72.2)

98.1(96.1–99.0)

33.3 (12.8–63.1)

98.6 (96.8–99.4)

–

Jollis [35]

1993

USA

1788

ACD

Registry

9

Any

36. 0 (33.8–38.3)

96.0 (95.6–96.4)

59.2 (56.3–62.2)

90.3 (89.7–90.8)

0.39

Li [57]

2011

USA

477

ACD a + registry

Registry

9

Any

27.9 (23.9–32.1)

75.6 (72.0–79.0)

47.8 (41.8–53.9)

56.7 (53.1–60.2)

–

Best alg

13.0 (10.1–16.4)

98.0 (96.5–99.0)

83.8 (73.4–91.3)

–

–

Roger [45]

2004

USA

658

HDD

MR

9

1ry

–

–

82.0 (81.1–82.9)

–

–

McCullough [41]

2002

USA

200

ACD

MR

9

Study alg

–

–

73.8

–

–

Frolova [30]

2015

Canada

733

HDDb

MR

10

Any

92.8 (90.6–94.5)

20.5 (12.0–31.6)

92.1 (90.0–94.0)

22.1 (12.9–33.8)

–

1ry

73.9 (70.6–77.1)

71.2 (59.4–81.2)

96.3 (94.4–97.7)

21.4 (16.4–27.1)

–

Juurlink [79]

2006

Canada

482

HDD

MR re-coding

10

1ry

79.0 (75.0–83.0)

99.5 (99.4–99.6)

85.0 (82.0–89.0)

99.3 (99.1–99.4)

0.82 (0.79–0.84)

Austin [24]

2002

Canada

5475

HDD

Registry

9

1ry

58.5 (57.2–59.8)

96.8 (96.6–96.9)

65.1 (63.8–66.5)

95.8 (95.6–96.0)

0.58

Lee [38]

2005

Canada

1808

HDD

MR

9

1ry

–

–

94.3 (93.1–95.4)

–

–

Blackburn [25]

2011

Canada

345

HDD

MR

9

1ry

–

–

73.8

–

–

10

1ry

–

–

84.5

–

–

Cozzolino [27]

2019

Italy

124

HDD

MR

9

1ry

96.0 (91.0–99.0)

90.0 (81.0–96.0)

94.0 (88.0–97.0)

93.0 (85.0–98.0)

–

Fonseca [29]

2008

Portugal

168

HDD

MR

9

Any

77.4 (70.3–83.5)

59.1 (46.3–71.0)

82.8 (76.0–88.4)

50.6 (39.0–62.2)

–

Bosco-Levy [26]

2019

France

229

HDD

MR

10

Any

64.2 (58.0–70.4)

–

60.5 (53.7–67.3)

–

–

Mahonen [39]

2013

Finland

313

Multiplea

MR

8, 9, 10

Study alg.

48.5 (42.9–54.2)

99.7 (99.5–99.8)

85.9 (79.7–90.5)

85.9 (79.7–90.5)

–

Merry [42]

2009

Netherlands

154

HDD

Registry

9

Any

43.0 (35.3–51.2)

99.9 (99.9–100.0)

80.0 (69.7–87.6)

99.6 (99.5–99.7)

–

Ingelsson [34]

2005

Sweden

321

HDD

MR

8, 9, 10

1ry

–

–

95.0 (89.6–97.8)

–

–

Any

–

–

81.7 (76.9–85.7)

–

–

Schaufelberger [46]

2020

Sweden

911

HDD

MR

10

Any

–

–

94.4

–

–

Thygesen [54]

2011

Denmark

50

HDD

MR

10

1ry

–

–

100 (92.9–100)

–

–

Mard [40]

2010

Denmark

637

HDD

MR

10

Any

–

–

84.0 (81.2–86.6)

–

–

Delekta [51]

2007

Denmark

418

HDD

MR

10

Any

–

–

83.6 (80.1–86.7)

–

–

Kümler [37]

2008

Denmark

429

HDD

Study doctorc

10

Any

29.4 (25.1–33.9)

98.9 (98.5–99.3)

80.8 (73.7–86.6)

90.0 (88.9–91.1)

–

Sundbøll [53]

2016

Denmark

100

HDD

MR

8,10

1st any/1ry

–

–

76.0 (66.0-83.0)

–

–

Pfister [52]

2013

UK

379

HDD

MR

10

Any

–

–

95.7

–

–

Khand [36]

2005

UK

216

HDD

MR

10

1–6

–

–

86.7 (82.4–90.1)

–

–

Teng [48]

2008

Australia

1001

HDD

MR

9, 10

1ry

–

–

98.8

–

–

Ono [43]

2020

Japan

5404

ACD

EMR

10

1ry

–

–

95.7 (95.2–96.2)

–

–

  1. Studies grouped by country and ordered by sensitivity or PPV (highest to lowest). Some studies contribute > 1 algorithm if assessing codes in > 1 position, or different ICD versions. See Table 1 in Additional file 1 for details of the algorithms used
  2. ACD indicates administrative claims data where the data source is specified as claims data by study, Alg. algorithm, CI confidence interval, EMR electronic medical record, GS gold standard, HDD hospital discharge data, HF heart failure, ICD International Classification of Diseases, MR medical records, Pos. position, PPV positive predictive value, NPV negative predictive value, RCD routinely collected data, Val. validation cohort
  3. aTwo or more data sources
  4. bRCD sources highlighted in bold from one country all used the same (or similarly structured) RCD source
  5. cProspective history and examination by study doctor
Back to article page

Systematic Reviews

ISSN: 2046-4053