Cervical cancer is the third most common cancer in women and while more than 85% of the cases occur in the developing world the burden of disease is also apparent in low-income, minority and low literacy populations of the United States [1-3]. In the United States, the annual incidence of cervical cancer is approximately 12,000 and over half of these cases occur in women who have never had a Pap smear, and an additional 20-30% of cases occur in women who have not had a Pap smear in the proceeding five years [4,5]. Although the Pap smear is the most widely used and accepted screening tool, it requires strict follow up to be effective. Patient communication, understanding, and health literacy are some of several structural barriers that are key elements of adherence to this screening test [6,7]. While studies show that educational interventions promote utilization of pap smears and adherence to testing, there is a paucity of published data regarding the efficacy of available educational material [8]. Our aim was to evaluate the ACOG education pamphlet Understanding Abnormal Pap Tests Results to determine if it was health literate for women receiving care at a university-based Medicaid Gynecology clinic in Manhattan, where the patient population is pre-dominately Hispanic and African American (Figure 1).

Figure 1: Patient Flow Diagram.

This randomized controlled trial was reviewed and approved by the Institutional Review Board (IRB) at Mount Sinai School of Medicine. A waiver of documentation of informed consent was obtained from the IRB. All female patients 18 years and older presenting to the Mount Sinai Hospital Medicaid clinic for a gynecology appointment were included. All patients who reported they could not read English were excluded. In order to randomize our subjects into our experimental group which provided the pap smear pamphlet, and our control group which provided a healthy living pamphlet, the Principal Investigator (PI) who was not involved with patient recruitment used a computer generated simple randomization scheme which assigned a number to either experimental or control group. A research student, also not involved with patient recruitment, sequentially numbered opaque envelopes 1 through 50, inserted the corresponding pamphlet according to the randomization scheme, and then sealed the envelope. In order to avoid visibility of the packet contents, the pamphlets were placed in smaller opaque envelopes within the larger envelope. Also included in each envelope were an instruction sheet, a pre-test survey, and a post-test survey. All contents of the envelopes were blinded to the investigators recruiting subjects and to the subjects themselves. Once all envelopes were created, we approached patients presenting to the Medicaid clinic for gynecology appointments and asked if they were interested in participating in our study. If they met our inclusion criteria and agreed to participate, information about the study was provided and verbal consent was obtained. The subject was then presented with the sealed envelope and instructed to open the envelope, read the instruction sheet, take the 5 question pre-test survey, read the pamphlet, and take the 5 question post-test survey. Figure 2 depicts the survey questions used on the pre- and post-tests. The questions specifically addressed the definition of a pap smear, an abnormal pap smear, and the basic management of abnormal pap smears. Once subjects completed their surveys, they had the option of retaining the pamphlet or returning it with their surveys. Investigators collected all completed surveys and scored them based on a 100% scale, with each question worth 20%. Patients’ demographic information obtained from the surveys was also recorded. A Chi-Square test was used to compare levels of education and racial distributions between control and experimental groups. A T-test was used to compare mean age between groups. The mean difference between pre and post pamphlet test scores was computed and tested for difference from zero within each group using a paired t-test. All statistical analyses were performed using SAS Version 9.2. All hypothesis testing was conducted at the 5% level of significance. We assumed a 20% difference between pre- and post-test scores, with post-test showing an improvement in pap smear knowledge. In order for our study to achieve 80% power and detect this 20% difference in test score, we needed at least 15 patients. Statistical significance was set at P value 0.05.

Figure 2: Pre- and Post-Test Survey Questions.

From December 2011 through March 2012 a total of 50 patients were recruited for participation in this study. Of the patients recruited to participate, 1 was excluded because the pre-test and post-test were not properly completed and left blank by the participant and 2 refused to participate. The remaining 47 results were included for evaluation, 22 were randomly assigned to the control group and 25 to the experimental group. Baseline characteristics are shown in Table 1. Median age was 32.40 and 30.18 in the control and experimental group respectively. The level of education among the two groups was also similar; with approximately 50% of participants in both groups having at least some level of college education (59% in the control group and 56% in the experimental group).

Table 1: Patient characteristics.

The pre-test and post-test surveys were calculated for each participant. The baseline scores were similar among the two groups, 89.09% in the control group and 84.80% in the experimental group. A statistically significant difference was noted between pre-test and post-test scores among the experimental group. In this group, patients improved their scores by 10.4% from baseline [p=0.0002]. In contrast, there was no statistically significant difference in score among patients in the control group between the pre-test and post-test results. A 1.82% decrease in score was noted, however this decrease was not found to be statistically significant [p=0.5043] (Table 2).

Table 2: Pre-Test and Post-Test scores and outcomes.

Health literacy is a major factor in a patient’s ability to understand and interpret health information and in their ability to make health care decisions [9-11]. In an effort to minimize the negative effects of low health literacy, patient education has become a primary focus of preventive health, and it is important to evaluate the efficacy of existing educational materials to insure that it is health literate for patients seen in hospital centers and clinics across the nation. Ongoing evaluation of such materials has been done in the form of readability calculators and validated indexes [12]. Because the cervical cancer burden of the United States weighs more heavily on lower socioeconomic and minority patients the disparities seen in the incidence of cervical cancer are likely due in part to varied levels of health literacy [2]. Review of the literature revealed that ACOG printed patient education materials have been assessed and found to be written at a higher reading level than believed to be acceptable when writing to a general audience [13,14]. Various patient educational modalities have been evaluated, and a recent systematic review assessing the efficacy of printed versus multimedia educational material found that the two modalities are equivalent [15]. However, to date, no other study has evaluated the ACOG printed patient education material for health literacy and comprehension in low-income minority populations in the United States as our study does.

In our study, we sought to evaluate a low literacy population and found that our patients’ baseline understanding of Pap tests was higher than anticipated. This higher level of understanding may be attributed to the education level of the patients evaluated, or related to their exposure to multiple modalities of preventative health education that were not controlled for in this study. Our results may not apply to clinic settings serving patients with lower levels of education. The limitation of this study is the lack of a validated survey tool. However, strengths include the large sample size, prospective randomized design and the similar baseline demographics of the control and experimental groups. Future aims are to evaluate ACOG reading materials in patients with lower levels of education and to develop a validated tool to examine other aspects of reading material that may affect comprehension such as graphics, layout, and culture appropriateness.

Our data suggests that the ACOG Understanding Abnormal Pap Test Results pamphlet is an effective education tool and increases patient understanding of abnormal pap smears. The increase in post test scores in the experimental group reflects the efficacy of the ACOG educational pamphlet in increasing patient understanding of abnormal pap smears these findings should encourage providers to utilize these pamphlets as a means of educating patients in high volume university-based Medicaid clinics. Although the complexities of access to healthcare among minority populations are multifactorial, the role of the healthcare provider as educator and the use of patient education materials such as the ACOG pap smear pamphlet, may improve health literacy and health outcomes.