Readability Assessment of Internet-Based Patient Education Materials on Anticoagulation Therapy

Background: Since the advent of the Web and media, there has been a significant change in the practice of Medicine. The physician is no longer the sole custodian of medical knowledge. A substantial amount of consumer health-related information is available on live streaming, other Internet sources and social media. Nearly two-thirds of American adults (65%) use social networking sites, up from 7% when Pew Research Center began systematically tracking social media usage in 2005. Pew Research reports have documented in great detail how the rise of social media has affected such things as work, politics and political deliberation, communications patterns around the globe, as well as the way people get and share information about health [1]. Studies suggest that consumer comprehension may be compromised if content exceeds what an average American can easily understand. The average reading level in the United States is 8 grade. The National Institutes of Health and the US Department of Health and Human Services (USDHHS) recommend that patient education materials (PEM’s) be written at or below the 6 grade level [2,3]. Objective: To determine the readability level of published materials likely to be encountered by a patient following a Google search of the phrase “blood thinners”, a popular synonym of anticoagulant. Study Design: Analysis of Internet-based PEMs on anticoagulation therapy. Methods: PEMs from the first 12 websites encountered on a Google search of the phrase, “blood thinners,” were downloaded and assessed for readability using 9 different indices: Flesch Reading Ease (FRE), Gunning Fog Index (GFI), Flesch-Kincaid Grade Level (FKGL), Coleman-Liau


Introduction
Anticoagulants (blood thinners) are the mainstay therapy for acute and long-term treatment and prevention of numerous inheritable and acquired thrombo-embolic disorders Internet proposes an easy-to-use, universal access to information and provides various possibilities to find the latest up-to-date, barrier free information that is independent of location and time.
Interactive services like online self-help-groups, chats with experts and forums on special health topics can support active coping and social support in a virtual community by anonymous contact. 71.7 percent of households reported accessing the Internet in 2011, up from 18.0 percent in 1997 (the first year the Census Bureau asked about Internet use) and 54.7 percent in 2003 [5].
The term "health literacy," as described by the Joint Commission (formerly the Joint Commission on Accreditation of Health Care Organizations), refers to "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions."2 The impact of literacy on health is significant. Adults with low literacy skills have a poorer health status, 18 [5,6]. The United States Department of Health and Human Services (USDHHS) resolved that material is considered "easy to read" only if written below a 6 th -grade level. Material between the 7 th and 9 th grade levels is viewed as "average difficulty," and material above the 9 th -grade level is regarded as "difficult [7]. Readability is a measure of the grade level necessary for an individual to adequately comprehend a written document [8]

Materials and Methods
In July 2016, the key phrase, "blood thinners," was typed into the Google search engine, and all of the websites that appeared The material from each of the 69 WebPages was copied and pasted into a separate Microsoft Word document. The documents were then scrutinized for any text that was not related to anticoagulation therapy. Examples include author information, sponsor information, disclaimers, and hyperlinks to other websites, copyright notices, references, citations, website information, and URL's. These types of text were then deleted from the document to prevent skewing of the readability statistics. Each Microsoft Word document was then subjected to readability analysis. Each page was copied and pasted into an online readability assessment tool www.readabilityformulas.com, to obtain 9 different readability Estimate Graph (REG), as well as the average grade level across all 9 calculators. Each of these 9 indices assesses the documents using different variables and formulas to generate an estimated reading level. These variables and formulas are described in Table 1.  The FRE, FKGL, GFI, CLI, SMOG, ARI, and LWF provide numerical reading scores via mathematical formulas, while the FRG and REG provide graphical representation of the readability.
a) The FRE is a numerical score between 0 and 100, with a higher number indicating a more readable document. It is influenced by average sentence length and average syllables per word [7]. Table 2 breaks down this scale to correlate the numerical values with a level of difficulty and associated grade level [8,9].
b) The FKGL assesses the same variables as the FRE but generates a grade level rather than a numerical score [10].
c) The GFI calculates a grade level based on average sentence length and percentage of words with more than 3 syllables, i.e. complex words.10 (4) The CLI generates a grade level based on the average numbers of sentences and characters per 100 words [11].
d) The SMOG Formula uses the numbers of sentences and complex words to generate a grade level [12].
e) The ARI calculates a grade level using the average word length and average sentence length [13].
f) The LWF uses the numbers of complex words, easy words (2 syllables or less), and sentences to generate a grade level [14].
g) The FRG generates a graph using the average numbers of syllables and sentences per 100 words, where the two lines intersect indicates the reading level of the document [15].
h) The REG uses the average numbers of sentences and long words (6 or more characters) per 100 words to generate a grade level via an intersecting point on a graph, similarly to the FRG [16]. The REG tends to estimate the grade level of a document several points below the FRG and all of the other readability indices. For this reason, medical researchers do not use it as frequently [17] of these 9 tools, the FRE, SMOG, GFI, and FRG are the most widely used readability calculators by medical experts, the other scales being referenced less often [9,17]. We felt that a composite average of multiple scales would be a more accurate approach to determining the true grade levels of these documents than any one or two scales used alone.
The readability data were then analyzed using the endmemo

Results
A total of 69 WebPages from 12 parent websites were analyzed for their level of readability using 9 different readability indices.      that these sites were authored at difficulty levels above 8th grade, frequently reaching readabilities of 10 th grade and higher. The REG calculations showed that most of the sites fell around the 6 th -7 th grade, but this graph has been shown to underestimate readability several levels below the other scales [21]. Because the REG uses word length and the number of sentences to compute its results rather than syllable number and sentence length (which are better predictors of readability), it tends to lead to falsely low readability results [22].         Patients taking anticoagulants who are at risk for bleeding and thrombosis need to learn and understand their condition.

Discussion
As increasing numbers of elderly patients with atrial fibrillation receive warfarin and other NOAC drugs, it is imperative to device an effective communication. Patient information should be written at an appropriate reading level, and its readability could be determined by using the various readability formulas. The National Work Group on Literacy and Health also recommends that material be written at or below the 6 th -grade level, [24-30] because material written at higher levels is less likely to be read or understood. Developing patient information at a low readability level is necessary but not sufficient to improve comprehension. Our study suggests that health information sites on anticoagulation therapy need to be rewritten at more understandable levels. There are many ways that these sites can be altered to bring their readability scores down ( Figure 1 and Figure 2). First of all, many of the writings used a large number of medical terms; using smaller, less scientific words has the potential to drop many of the readability scores substantially.
Limiting sentence length and reducing the complexity of sentence structure can also reduce scores and make the document far more easily understood. Other methods of communication, and written information that uses figures, pictograms, large font, and other characteristics, may also improve comprehension [31].

Figure 2:
Comparing websites on the most popular direct thrombin inhibitors Y axis-Grade level X axis-Website comparison-Xarelto/eliquis/Pradaxa.

Conclusion
The Internet-based patient education materials on anticoagulation therapy are written at levels far above the recommendation of 6th grade. Due to a wide-spread pool of information, which can be personalized, Internet can enhance health literacy, health related knowledge and support people to become responsible for their own health. Hence, much attention needs to be paid to readability of these health education materials. Current Internet-based PEMs on anticoagulation therapy should be revised to make them more easily understood by the average American.