Syncope is a common cause of Emergency Department (ED) presentation worldwide, accounting for 1-3% visits and affecting 40% of the population at least once in their lifetime [1,2]. It is a syndrome characterized by transient loss of consciousness (T-LOC) associated with inability to maintain postural tone, with rapid onset, short duration and spontaneous, complete recovery [3,4]. This is consequent upon transient, self-limited global cerebral hypoperfusion due to cardiac or neurological causes [5]. The role of the Emergency Physician (EP) is to identify subjects at high risk of major adverse events, most commonly those with cardiac disease who benefit from inpatient evaluation, whilst minimizing admission of low risk presentations. Several clinical decision tools (CDTs) have been developed to assist EP’s in this process, but are not in widespread use [6]. The San Francisco Syncope Rule (SFSR) and the Osservatorio Epidemiologico per la Sincope nel Lazio (OESIL) Score have been externally validated with variable performance. They have been applied in the Emergency Department setting to predict short (7 and 30 days) and less commonly long-term (1 year) outcomes [7-9]. The ROSE (Risk stratification Of Syncope in the Emergency Department) Rule is the first CDT to incorporate the biochemical biomarker serum brain natriuretic peptide (BNP) [10]. This biomarker may improve the detection of syncope secondary to structural heart disease, but it is not universally available in the ED [11,12]. This has seen age substituted for BNP levels, with age over 65 added as a risk factor for adverse outcomes, creating the “ROSE-65” in this study [13,14].

The reliability of these CDTs has been the subject of recent reviews and meta-analyses, however none has demonstrated clear superiority with sensitivities / specificities for adverse outcomes ranging 0.61-0.88 / 0.51–0.60 respectively [15-17]. Development of a single tool to unify early management of these patients is a current priority in syncope research [4].

The aim of this study was to evaluate the reliability of the ROSE Rule (minus BNP) which has not been externally validated, and to compare its performance to clinical care, the OESIL and SFSR CDT’s to predict adverse outcome in patients presenting with syncope at 1-week, 1-month and 1-year.

Design and setting

This was a single center retrospective observational cohort study performed in the emergency department (ED) of an inner city tertiary hospital with an annual census of 130,000 adult attendances. There were no decision tools or guidelines for management of syncope in place.

Study population and data collection

Patients aged 16 years or older presenting to the ED with syncope (defined as T-LOC with or without prodromal symptoms characterized by short duration and spontaneous full recovery) were eligible for the study. Exclusion criteria were as follows: age <16, seizure, persisting LOC (>1 hour), any focal neurological deficit, alcohol/drug related collapse, post-traumatic LOC or hypoglycemia. During the study period a consecutive, convenience sample was identified by duty physicians (all grades of staff were included with staff ranging from second year trainees to consultant/attending senior doctors), during working hours (08:00-22:00) and details recorded. Final diagnosis was as documented in the medical record. After the clinical episode the study team independently reviewed all data and enrolled those eligible by inclusion criteria. Clinical, laboratory, and outcome data was extracted for analysis including: demographics, ED diagnosis, disposition (admitted to hospital/ admitted to Clinical Decision Unit (CDU)/ discharged from ED); a priori defined high risk history features (no prodrome/ palpitations/ chest pain/ shortness of breath/ headache/ syncope during exertion or while supine/ reported gastro-intestinal bleeding and history of valvular heart disease/ congestive cardiac failure/ ischemic heart disease); and high risk examination features (injury resulting in fracture/ meleana or faecal occult blood positive on rectal examination /new cardiac murmur/ systolic blood pressure <90 mmHg/ postural hypotension > drop of 20 mmHg or a value <90 mmHg/ SpO2<94% on air / heart rate <50 bpm pre-hospital or in ED). All data was collected by the lead author (RB) into a purpose built database and reviewed by the senior author (TH). All ECGs were independently interpreted by 2 cardiologists blinded to the clinical details. We used the dedicated database to calculate the scores according to OESIL, SFSR and ROSE Rule for each CDT to reduce personal interpretation bias.

Follow up

All patients were followed up by telephone and by searching the Trust wide electronic patient record system to identify adverse outcome data at 1-week, 1-month and 1-year. Where no follow data was found, the General Practitioner was contacted. The primary end-point was the combination of adverse outcomes (AO) plus all-cause mortality at each follow up stage. Adverse outcome were defined a priori as one of the following events: acute myocardial infarction (STEMI/ NSTEMI defined by discharge diagnosis), cardiac arrest, arrhythmia requiring treatment, pacemaker or ICD implantation, cardiac stent insertion, cerebrovascular accident, haemorrhage requiring transfusion/ endoscopy or surgery, non-traumatic intracranial (IC) bleeding or pulmonary embolism (PE).

Retrospective scoring of OESIL, SFSR, ROSE (minus BNP) and ROSE-65 were performed for each timeline (7 days, 1 month and 1 year).

The OESIL score offers one point for each of: abnormal ECG (see appendix1), a previous history of cardiovascular diseases, absence of prodromal symptoms, and age greater than 65 years (score range 0-4) [9]. In this study we defined a positive OESIL score (high risk for adverse outcome) if score > 1. SFSR is deemed positive and hence high risk, if any of the following are present: a history of congestive heart failure, hematocrit lower than 30%, abnormal ECG (see appendix 1), a complaint of shortness of breath, and systolic blood pressure lower than 90 mm Hg). The ROSE rule is considered positive and hence high risk, in the presence of any of: BNP level ≥ 300 pg/ml or age >65(ROSE 65), bradycardia ≤50 in ED or pre-hospital, rectal examination showing fecal occult blood, anemia with hemoglobin ≤90 g/l, chest pain associated with syncope, abnormal ECG or SpO2 ≤94% on room air.

Ten patients were lost to follow-up as we were unable to contact them directly or via their general practitioner and there was no record of any further ED attendance. All were young (<35 years) with normal ECGs, a diagnosis of vaso-vagal syncope and discharged directly from the ED. Our population includes a high proportion of visitors to the UK, transient workers and is characterized by a young, mobile population.

Ethics review

The study was reviewed by the National Research and Ethics Service (NRES ref 04/01). As the study was retrospective, observational, collected no patient identifiable data, involved data being stored on a dedicated secure Trust computer, included no deviation from usual practice and no involved no interventions is was classified as audit. It was registered according to local and National Health Service guidance with our Trust and approved by our Researched and Development Department.

Statistical analysis

Baseline characteristics of enrolled patients were reported as descriptive statistics with mean, frequency, percentage and standard deviation. Differences among variables were evaluated by Student t test, χ² test, and the Fisher exact test. P<0.05 (2-tailed) was considered significant.

For each clinical decision rule and for clinical judgment, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and positive and negative likelihood ratios (PLR and NLR respectively) were calculated to identify patients who experienced adverse outcomes at each follow-up stage. Data was entered into a Microsoft Access 2007 database (Microsoft Corporation, Redmont, Washington USA), exported to Excel, and analyzed using STATA version 12 (StataCorp LP©, Texas, USA).

Between July and December 2009, 148 patients were identified and of whom 120 met inclusion criteria with data available, as outlined in Figure 1. Demographic features and final diagnoses are presented in Table 1. There was no significant difference in age between males and females (p=0.17).

Figure 1: Flow diagram of the study scheme and enrolled population. Ten patients could not be traced by telephone or via their GP with no record of re-attendance to the ED at 1-year . These patients were thus excluded from analysis as we were unable to determine AOs.

Table 1: Demographics, diagnosis and disposition of analysed patients. Adverse outcomes (AO) were defined if one of the following were present: acute myocardial
infarction (STEMI/ NSTEMI), cardiac arrest, arrhythmia requiring treatment, pacemaker or ICD implantation, cardiac stenting, cerebrovascular accident, haemorrhage
requiring transfusion/ endoscopy or surgery, non-traumatic intra cranial bleeding or pulmonary embolism. All cause death and AO are reported at 1-year follow up.

Overall, AO and all causes of death occurred in 5 (4%) patients at 1-week, cumulatively 6 (5%) at 1-month and 15 (12.5%) at 1-year follow-up. The causes of death were as follows: two died of stroke, one of congestive cardiac failure, one of pneumonia, and one died of end stage renal failure. Adverse events included one patient who suffered cardiac arrest with subsequent ICD insertion, one had an episode of supra-ventricular tachycardia, one had gastro-intestinal bleeding and four patients required pacemaker implantation and one had an ischemic stroke, one a subarachnoid hemorrhage and one required coronary stenting.

Disposition by CDT is reported in Table 2. Outcomes by CDT at each follow up time point are reported in Table 3.

Table 2: Disposition after ED assessment reported for each CDT.

Table 3: Number and percentage of patients discharged from ED (including those sent to and discharged from CDU) and those admitted to hospital, reporting adverse events (AO). See text for definition of postive/negative score for each clinical decision rule.

Potential saved admissions the performance of clinical practice and each CDT (OESIL, SFSR, ROSE Rule and ROSE-65), are reported in Tables 4 and 5.

Table 4: Below the number of potential prevented admissions and missed AO for each score at each follow-up time point as compared to clinical care. See text for definition of postive/negative score for each clinical decision rule.

Table 5: Performance of clinical judgment (Clinical Care), OESIL (Osservatorio Epidemiologico per la Sincope nel Lazio) >1, SFSR (San Francisco Syncope Rule), ROSE (ROSE Rule minus BNP), ROSE-65 (ROSE Rule substituting age>65y for BNP) at each stage of follow-up (1-week, 1-month and 1-year) to identify AOs. P value was calculated using Fisher exact test comparing AO against each rule.

Overall ROSE outperformed ROSE-65 at 1 week: sensitivity 80% for both and specificity 81% versus 64% respectively. Similar results were found at 1 month with sensitivity 83% for scores, specificity 82% and 64% for ROSE and ROSE-65 respectively. Both performed better than clinical judgment alone, which had similar a sensitivity 80% at 1-week, 83% at 1-month, and a specificity of 59% both 1-week and 1-month. ROSE-65 performed best at 1-year time lines with a sensitivity 93% and specificity 70%.

To our knowledge, this is the first study to externally validate the ROSE Rule, albeit in a modified form. We conducted this study to compare our clinical practice to CDTs used in the assessment of patients presenting to ED with syncope. We enrolled and followed up 120 patients at 1-week, 1-month and 1-year and compared current clinical care to admit/discharge against the OESIL, SFSR, ROSE (minus BNP) and ‘ROSE-65’ CDT’s. The potential number of admissions saved by using each CDT and the potential number of AO identified and missed were calculated.

Overall, 12.5% (n=15) of our study cohort had an AO at one year, of which 33.3% (n=5) occurred within 1-week. Other than OESIL>1, which showed the lowest sensitivity (0.40), all CDT’s performed with similar sensitivity at 1-week and 1-month but the ROSE (minus BNP) had significantly higher specificity at both time points, (see Table 5). Furthermore using the ROSE CDT, 26 (21.6%) unnecessary admissions would have been prevented, but at the cost of discharging 1 patient who had AO at 1-week and no further AO’s at 1-month. This is meaningful as the assessment of short-term (1-week/1-month) outcomes has been recently recommended as the preferred outcome time frame for CDT development in syncope research [4,14,18,19]. Thus the ROSE CDT combined identifying the highest number of potentially preventable admissions with the highest sensitivity of identifying AO’s as compared to clinical care.

Application of ROSE-65 would have resulted in a sensitivity and specificity of 93.3% and 70.5% respectively at 1-year follow up to identify AOs. This would have prevented 6 (5%) admissions but missed 1 (0.8%) AO. It is possible that serum BNP measurement may have improved performance of both ROSE and ROSE-65.

Similar to previous studies, vasovagal syncope occurred most frequently, but the proportion of patients with this diagnosis (59.2 %) was higher than data reported [20]. This could be explained by the lower mean age (49.5 years) of our ethnically diverse cohort [18]. Increased female prevalence is in keeping with other work [21,22]. However overall AO remained in line with the review meta-analysis study of Solbiati et al., occurring in 5-16% of the syncope patient population [23]. SFSR and the ROSE performed similarly to previous validation studies with higher sensitivity than clinical judgment in predicting adverse events [24,25]. Conversely at 1-week and 1-month follow up OESIL>1 showed similar specificity (around 75%) but lower sensitivity as compared to previous findings (40% in the current study versus 88% found by Di Paola et al. [26] and Reed et al. [27]. ROSE-65 performed better than the ROSE rule at 1-year, demonstrating the best performance with NPV 98%, sensitivity 93%, specificity 70% (p<0,001). This may reflect the fact that elderly patients are often more frail with multiple co-morbidities, therefore predisposed to adverse outcomes. Only OESIL and ROSE-65 CDT use age as an AO predictor [28].

Despite a sensitivity of 80% and NPV of 98% at 1-week and 1-month (see Table 5), clinical care resulted in 5 discharged patients having adverse outcomes. In line with the findings of previous studies, application of OESIL>1 and SFSR were found to be safe tools to assess admission risk/benefit as compared to clinical judgment. Their use would have prevented 21 admissions and led to 2 further admissions respectively while missing 1 AO each [6,29].

We were unable to identify any previous external validation study of the ROSE CDT. Compared to the original work by Reed et al. our results showed: sensitivity of 83.3% (vs 87.2%), specificity 82.5% (vs 65.5%) and a NPV of 98.9% (vs 98.5%) at 1-month follow up; sensitivity of 66.7% (vs 71.6%), a specificity 85.7% (vs 71.1 %) at 1-year follow up [10,30]. In their cohort of 529 patients, the ROSE rule would have prevented fewer admissions than in our study (80 admissions prevented, 1 every 6.6 attendances; as compared to our study with 26 admissions prevented, 1 every 4.6 attendances) but missed fewer AOs (5, 1/105 compared with 3/120, 1 in 40 in our group). The modified ‘ROSE-65’ CDT performed almost as well as ROSE (minus BNP) at 1-week and 1-month, and better at 1-year.

Comparing our data with previous meta-analyses, ROSE and ROSE- 65 performed better than the other CDT’s and clinical care at short term follow up. The meta-analyses report sensitivity and specificity for clinical judgment to predict AOs to be 95.0% and 55.0% respectively at 10-days, whereas in our study we found sensitivity 80.0% and specificity 59.1% at 1-week. Previous studies reported an OESIL>1 to have a higher sensitivity (78.0% at 10-days vs 40.0% at 1-week) but lower specificity (56.0% 10-days vs 75.7% at 1-week) when compared to our findings. For the SFSR we found a similar performance with sensitivity 76.0% and specificity 56.0% at 10 days compared to sensitivity 80.0% and specificity 57.4% in our study [13-15].

In conclusion, applying ROSE Rule (even minus BNP) would have resulted not only in using the CDT with the higher performance, but also would have saved the greatest number of admissions with resultant cost-saving implications [31]. Our data is drawn from a single center so generalization of the findings is limited; however our case mix is drawn from a highly diverse population.

This study included a small convenience sample. Patient enrollment was retrospective and based on emergency physician recorded data. No prospective, standardized data collection tool was utilized which may have resulted in imperfect data. We attempted to minimize the problems of retrospective data collection by identifying patients each day during predefined working hours. However some clinical information may not have been recorded. For example, ‘lack of prodrome’ may not have been documented and could account for lower performance of the OESIL Rule in this cohort. We attempted to minimize this risk by offering standardized teaching on syncope to all trainees as part of our standard rolling teaching program. Of the 139 eligible patients nine were excluded as ECGs were missing from medical records. This is a vital test in syncope assessment, recommended by national and international guidelines without which important aetiological factors may be overlooked and is required for all studied CDTs [3,32]. Therefore, where the ECG had been lost we were unable to include patients in the study. Furthermore 10 patients were not contactable post discharge either directly or via their GP and were thus lost to follow up. Potentially, some of this group may have suffered AO/ been admitted to other hospitals. However all were young (age <35) with a discharge diagnosis of vaso-vagal syncope, normal ECG and there was no record of repeated attendance or follow up at our institution. The ROSE CDT was assessed without the use of BNP and so was not applied as the authors intended. In addition, the modified ROSE-65 rule was calculated without any previous evidence of efficacy (derivation or validation). Further evidence will be required to verify its performance.

As syncope is considered a single clinical event in a particular moment and time, often without the opportunity to make a firm diagnosis, we cannot say with certainty that AOs, which occurred after the index presentation, were linked pathophysiologically to that specific episode [33].

The assessment of syncope remains a challenge for Emergency Physicians. CDTs have been developed to assess short, medium and long-term risk but are not widely adopted. The elderly population is often under-represented in studies which limits generalizability. This study suggests that SFR, ROSE (without BNP) and ROSE 65 had similar sensitivities (and so a similar short term ability to identify AOs) at 1-week and 1-month but the ROSE (without BNP) had a higher specificity, suggesting its use would be associated with fewer unnecessary admissions. The ROSE CDR also demonstrated a higher specificity at one year, but at the expense of a lower sensitivity. At 1-year ROSE 65 demonstrated similar sensitivity of the SFR and OESIL but had a higher specificity. However further evaluation with a large prospective validation cohort is required to define clinical use. No single CDT would have identified all AO’s even at one week and all would have admitted varying numbers of patients unnecessarily.

We acknowledge the cardiologists who reviewed the ECGs Dr Andrew Ludman MD, and Dr Oliver Guttman MD and Ms. Anna Springett who helped us with statistic analysis.