The Shapiro Score, or How “Facts Don’t Care About Blood Cultures.”

Does anyone remember signing out as residents, how there is always a line that goes something along the lines of “if fever, then culture.” As in fever always means there is an infection, or at least a suspicion of it. Of course, fever means inflammation but infection is a type of inflammation so why not get blood cultures? Well for one, when we get blood cultures, we are trying to catch a bacteremia, or bacteria in the blood. For another, we just suck at predicting who is actually bacteremic. As you can see, the rate of bacteremia in the cohorts I cite today range from 8-15%. This, of course, counts all comers who get blood cultures. As it turns out, there are tools out there that attempt to predict who, at minimum, is at low risk of bacteremia and thus, who can forego getting blood cultures. While you can take out your calculator (or MDCalc) and try to input these scores to decide as to whether or not order cultures, I hope you can notice a pattern as we go along. 

First thing first, we should probably get rid of the notion that fever means automatic ordering of blood cultures. A prospective cohort study of 363 patients hospitalized at a VA center found that fever, fever and leukocytosis, or fever and additional indications was not correlated with blood culture positivity (1). Those who had not received prior antibiotics had a higher likelihood ratio, although the ratio for those patients who had either persistent bacteremia or endocarditis was still higher in this cohort:

A post-hoc analysis of an observational cohort study (2) of just over 3500 patients found that, in 289 patients with positive blood cultures, 33% had a normal body temperature while 52% had a normal WBC (4-12 K/uL). 17.4% of patients with positive blood cultures had both a normal temperature and white count. While these studies evaluated different populations, relying on temperature or leukocytosis alone was not enough to predict blood culture positivity.

Enter, the Shapiro score.  This was a score that was derived in a prospective cohort of patients who presented to an emergency room (3). 3730 patients were enrolled, with ⅔ being used in a derivation cohort and ⅓ into a validation cohort. 13 predictors of bacteremia were identified:

Using this, a clinical prediction rule with points for each covariate was created (done by rounding the beta-coefficent to the nearest integer). Low risk patients (those with 0-1 points) had a 0.6% rate of bacteremia; intermediate risk patients (2-5 points) had a 6.8% rate of bacteremia; high risk patients (>6 points) had a 25.6% rate of bacteremia in the derivation cohort:

Using this risk stratification, a decision rule using major and minor criteria was created, with the presence of one major or 2 or more minor criteria indicating the need for a blood culture:

The sensitivity of this score was above 97% in both the derivation and validation cohorts, and the NPV being above 99% in both as well:

The high sensitivity suggests that it is very useful for identifying low risk patients, though it is helpful to pay attention as to what risk factors automatically lead to a blood culture order recommendation. A Danish cohort study validated the score in 1526 patients who presented to the ED (4). The rate of bacteremia was 6.9% (105), with patients with true bacteremia being matched with 315 patients with negative cultures. The sensitivity and specificity of the rule was 94% (95% CI 88-98) and 48% (95% 42-53%) respectively:

The Tokuda score is a similar score that attempts to evaluate predictors of bacteremia. A prospective study of 526 patients used recursive partitioning analysis to build a prediction algorithm (5), with univariate analysis finding that old age, high body temperature, high CRP values, and chills were associated with bacteremia. Two clinical scenarios were provided by the recursive partitioning analysis, with the first one using pulse, chills, temperature, low blood pressure, and low-risk site (i.e.acute pharyngitis, acute  bronchitis, infectious diarrhea, acute viral syndromes, pelvic inflammatory disease,  acute otitis media, acute sinusitis, and non-infectious process) and the second one using chills, CRP, and low risk site:

The risk of bacteremia in the low-risk group was less than 2% in each scenario:

The sensitivity of the second scenario was higher, allowing it as a better tool to rule out the need for blood cultures:

A retrospective study of ED patients evaluated 8177 patients and found that age >55, liver disease, chronic kidney disease, SOT, history of chills, and fever were associated with positive blood cultures (6):

An observational cohort compared 5 clinical scores (including the Shapiro and the Tokuda scores) and 6 biomarkers for their ability to predict blood culture positivity in 1083 patients who presented to the ED (7). The rate of bacteremia was 9.6%. The Shapiro score performed the best out of the clinical scores, with an AUC of 0.729, with the Tokuda score II coming in at second (AUC 0.665). The best biomarkers included PCT, NLR, and lymphocyte counts:

Combining both the Shapiro score and PCT yielded the best AUC compared to the combination of either NLR or lymphocyte count with the Shapiro score:

While the cut-offs of this are not discussed above, various cut-offs for both the Shapiro score and PCT were used, with the sensitivity for a Shapiro score of >2 and PCT >0.25 having a sensitivity of 98% (95% CI 93-99.8%)  and NPV of 99% (97-99.9%).

 
Other data comes from patients suspected of having community acquired pneumonia. A prospective study used data from the Medicare National Pneumonia Project which included over 39,000 patients with presumed pneumonia (8). Both a derivation and a validation cohort were used. SBP <90, hypothermia or hyperthermia, and tachycardia >125; BUN >30 mg/dL, sodium <130 and WBC <5K and 20K were independently associated with bacteremia in community acquired pneumonia in both the derivation and validation cohorts:

The risk of bacteremia in this study also included the use of prior antibiotics into its algorithm, with patients with low risk (see below) having a bacteremic rate of 2-3% in both cohorts:

A simplified model using liver disease and vital signs found that 86% of patients with bacteremia would have been detected, though it identified only 89% of patients with PSI of V:

One of the notable things is that the presence of hypoxemia, pleural effusion, or number of lobes involved in radiography were not associated with the presence of bacteremia, while the RR >30 was (OR 1.3, 95% CI 1.1-1.5) but was not included in the model. Another prospective study of CAP patients coming in through the ED evaluated 2422 patients to generate a prediction tool for bacteremia and randomized them into 2 groups to create a derivation cohort and a validation cohort (9). Multivariate analysis was performed to obtain candidate variables and the  model was developed using a regression coefficient-based scoring method. The rate of bacteremia overall in this cohort was 5.7%, with multivariate analysis finding that SBP <90, HR >125, temperature of <35C or >40C, WBC <4K or >12k, and CRP >17 being highly associated:

The cutoff for low risk was <5 points, with low risk patients having a rate of bacteremia <3% in both cohorts (AUC 0.75, 95% CI 0.68-0.83). This scoring tool was also applied to an external validation cohort that consisted of 1429 patients, with an AUC of 0.79 (95% CI 0.75-0.84). 

I know I went over several scores and algorithms, but I hope I made the point that just because someone is febrile it means they’re bacteremic. Quite the opposite, as noted in the Shapiro and Tokuda scores and in the CAP algorithms, extremes of temperature tend to be more strongly associated. Furthermore, it seems that certain comorbidities such as liver disease and the presence of central lines/risk of infective endocarditis (i.e. an IV drug user) also plays a role. One thing to note in the CAP studies was that a lot of the variables associated with bacteremia are essentially those for SIRS criteria. For instance, an early cohort study (10) of 270 unselected blood culture episodes found that all 11 episodes of true bacteremia had met SIRS criteria, though this meant 93% of patients with SIRS criteria were culture negative:

Furthermore, in 64 episodes of clinically important bacteremia, 95% (61/64) had met SIRS criteria, suggesting this could be used to forego blood cultures in certain patients:

A later study of a population of over 20,000 ED patients found SIRS criteria performed modestly when comparing a younger population (<65yo) to an older population (>75yo, adjusted AUC 0.60 vs 0.63, respectively, 11). Within this cohort, hyperthermia, tachycardia, and tachypnea were strongly predictive of bacteremia, with each variable performing better in the elderly population:

A small prospective study (12) compared 68 patients with bacteremia with 828 patients without and found that respiratory rate and body temperature were associated with bacteremia (aOR 5.42, 95% CI 1.13-25.9 and aOR 2.55, 95% CI 1.34-4.87). SIRS had a crude OR of 7.25 (95% 7.25, 95% CI 1.75-30.1) for bacteremia. This suggest that the so forgotten SIRS criteria can be used as a rough screening tool to rule out patients who do not need a blood culture, though their sensitivity is not as great as one would hope for.

In another interesting study that kind of went over my head, six different models using machine-learning found that a model using neural networks had the highest specificity (13). Another one using logistic regression was found to have the highest sensitivity of the models compared. That is not the important part. Univariate analysis found that presence of a central line (OR 1.87, 95% CI 1.67-2.09) and fever ( OR 1.5, 95% CI 1.28-1.75 if >38C, OR 1.61, 95% CI 1.11-2.38) were found to be the most significant factors associated with blood culture positivity. Further, qSOFA (OR 1.22, 95% CI 1.15-1.29) and SIRS positivity (1.18, 95% CI 1.15-1.29) were also associated with blood culture positivity. 

Despite their possible utility, a lot of these scores (many of which I did not go over) may not be worth your time. A review of 15 publications and over 59,000 patients found that, in general, many did not perform that well, with most of the AUC for the ROC ranging from 0.6-0.91 for both derivation and validation cohorts (14). Here, the authors defined the low-risk group as having a <3% rate of bacteremia and high risk as those with >30% risk.

The point here is not necessarily to tell you about the scores, but rather to give you a feel about the possible variables to consider when deciding when to get cultures. Fever or leukocytosis by themselves may not necessarily be enough to warrant you to look for a bacteremia. Fever in someone with liver disease or solid organ transplant, someone with high fevers, those who meet SIRS criteria or those with central lines may always warrant a blood culture. As things progress, it seems that computerized decision support systems may come into play. For instance, a multinational cohort study used the TREAT computerized decision support system as a tool to detect bacteremia, though the AUC-ROC was 0.68 for the one cohort while it was 0.70 for another (15). Notably, those patients who were deemed low risk by the system had a rate of bacteremia ranging from 1.3%-2.4%, suggesting this could be used as a tool to identify low risk patients and forego blood cultures. All told, one can use their clinical gestalt to rule out patients who need blood cultures, and one of the simplest one to use is the SIRS criteria along with risk factors for endocarditis like indwelling line. In the future, I’ll talk about things such as PCT, CRP, and eosinopenia and their utility for predicting bacteremia. 

References:

1. Linsenmeyer K, Gupta K, Strymish JM, Dhanani M, Brecher SM, Breu AC. Culture if spikes? Indications and yield of blood cultures in hospitalized medical patients. J Hosp Med. 2016 May;11(5):336-40. doi: 10.1002/jhm.2541. Epub 2016 Jan 13. PMID: 26762577.
2. Seigel TA, Cocchi MN, Salciccioli J, Shapiro NI, Howell M, Tang A, Donnino MW. Inadequacy of temperature and white blood cell count in predicting bacteremia in patients with suspected infection. J Emerg Med. 2012 Mar;42(3):254-9. doi: 10.1016/j.jemermed.2010.05.038. Epub 2010 Jul 31. PMID: 20674238.
3. Shapiro NI, Wolfe RE, Wright SB, Moore R, Bates DW. Who needs a blood culture? A prospectively derived and validated prediction rule. J Emerg Med. 2008 Oct;35(3):255-64. doi: 10.1016/j.jemermed.2008.04.001. Epub 2008 May 16. PMID: 18486413.
4. Jessen MK, Mackenhauer J, Hvass AM, Ellermann-Eriksen S, Skibsted S, Kirkegaard H, Schønheyder HC, Shapiro NI; CONSIDER Sepsis Network. Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med. 2016 Feb;23(1):44-9. doi: 10.1097/MEJ.0000000000000203. PMID: 25222426.
5. Tokuda Y, Miyasato H, Stein GH. A simple prediction algorithm for bacteraemia in patients with acute febrile illness. QJM. 2005 Nov;98(11):813-20. doi: 10.1093/qjmed/hci120. Epub 2005 Sep 20. PMID: 16174688.
6. Phungoen P, Lerdprawat N, Sawanyawisuth K, Chotmongkol V, Ienghong K, Sumritrin S, Apiratwarakul K. Clinical factors associated with bloodstream infection at the emergency department. BMC Emerg Med. 2021 Mar 12;21(1):30. doi: 10.1186/s12873-021-00426-2. PMID: 33711935; PMCID: PMC7953601.
7. Laukemann S, Kasper N, Kulkarni P, Steiner D, Rast AC, Kutz A, Felder S, Haubitz S, Faessler L, Huber A, Fux CA, Mueller B, Schuetz P. Can We Reduce Negative Blood Cultures With Clinical Scores and Blood Markers? Results From an Observational Cohort Study. Medicine (Baltimore). 2015 Dec;94(49):e2264. doi: 10.1097/MD.0000000000002264. Erratum in: Medicine (Baltimore). 2017 Feb 17;96(7):e6197. PMID: 26656373; PMCID: PMC5008518.
8. Metersky ML, Ma A, Bratzler DW, Houck PM. Predicting bacteremia in patients with community-acquired pneumonia. Am J Respir Crit Care Med. 2004 Feb 1;169(3):342-7. doi: 10.1164/rccm.200309-1248OC. Epub 2003 Nov 20. PMID: 14630621.
9. Lee J, Hwang SS, Kim K, Jo YH, Lee JH, Kim J, Rhee JE, Park C, Chung H, Jung JY. Bacteremia prediction model using a common clinical test in patients with community-acquired pneumonia. Am J Emerg Med. 2014 Jul;32(7):700-4. doi: 10.1016/j.ajem.2014.04.010. Epub 2014 Apr 18. PMID: 24856736.
10. Jones GR, Lowes JA. The systemic inflammatory response syndrome as a predictor of bacteraemia and outcome from sepsis. QJM. 1996 Jul;89(7):515-22. doi: 10.1093/qjmed/89.7.515. PMID: 8759492.
11. Chou HL, Han ST, Yeh CF, Tzeng IS, Hsieh TH, Wu CC, Kuan JT, Chen KF. Systemic inflammatory response syndrome is more associated with bacteremia in elderly patients with suspected sepsis in emergency departments. Medicine (Baltimore). 2016 Dec;95(49):e5634. doi: 10.1097/MD.0000000000005634. PMID: 27930596; PMCID: PMC5266068.
12. Leth RA, Forman BE, Kristensen B. Predicting bloodstream infection via systemic inflammatory response syndrome or biochemistry. J Emerg Med. 2013 Feb;44(2):550-7. doi: 10.1016/j.jemermed.2012.07.059. Epub 2012 Sep 19. PMID: 22999775.
13. Mahmoud E, Al Dhoayan M, Bosaeed M, Al Johani S, Arabi YM. Developing Machine-Learning Prediction Algorithm for Bacteremia in Admitted Patients. Infect Drug Resist. 2021;14:757-765. Published 2021 Feb 25. doi:10.2147/IDR.S293496
14. Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models–a systematic review. Clin Microbiol Infect. 2015 Apr;21(4):295-301. doi: 10.1016/j.cmi.2015.01.023. Epub 2015 Feb 10. PMID: 25677625.
15. Paul M, Andreassen S, Nielsen AD, Tacconelli E, Almanasreh N, Fraser A, Yahav D, Ram R, Leibovici L; TREAT Study Group. Prediction of bacteremia using TREAT, a computerized decision-support system. Clin Infect Dis. 2006 May 1;42(9):1274-82. doi: 10.1086/503034. Epub 2006 Mar 30. PMID: 16586387.