Research ArticleAbstract
Background: The important role of research in nursing professional development
increases the considerations to comparing clinical interventions, because they generate
nursing evidence and knowledge.
Objectives: The purpose of this study was to determine a new vision to efficacy,
comparative effectiveness and cost-effectiveness research in nursing.
Design: An integrative review and Delphi technique in combination.
Methods: Twenty-four papers enrolled to the review over 17102 initial papers
from Science Direct, PubMed and Google Scholar databases from 2000-2021. A one
round Delphi technique with 7 nursing experts was conducted to confirm or emphasize
on the collected data in the review.
Results: The review resulted in the characteristics of efficacy, comparative
effectiveness and cost-effectiveness research. Also nursing experts mentioned the
importance and existing obstacles to conduct effectiveness studies.
Conclusion: The main conclusion of conducting efficacy and effectiveness research
which produces complementary evidence is the reduction of the gap between theory
and practice in nursing and its professional development.
Keywords: Efficacy; Comparative Effectiveness; Cost-Effectiveness
Abbreviations: FCC: Federal Coordinating Council; IOM: Institute of Medicine of The National Academies; ARRA: American Recovery and Reinvestment Act; ACA: Affordable Care Act; DNP: Doctor of Nursing Practice
Introduction
Research possesses an important role in nursing practice
development which can provide an opportunity to develop
new knowledge and improve nursing care [1]. Recently, there
is an increased focus on research conducted to compare clinical
approaches [2]. Because the results of these studies are more
patient-centred and generalizable [3]. Also nursing profession
requires knowledge transfer from research to clinic [4] in order
to utilize the obtained evidence regarding to implementations and
interventions in practice. An important aim of clinical interventions
research is to respond health care problems or needs of patients.
Which can lead to produce clinical evidence and knowledge by
identifying the effectiveness of the desired intervention [5]. However,
nursing research activities have fewer focus on development of
nursing interventions, while finding effective solutions to patient problems should be the primary motivation of nursing and its
research community [5]. Therefore, it is necessary to mention on
a more detailed information rather than effectiveness itself, that
in which individuals and what circumstances the intervention
is effective. Because it can develop clinical knowledge produced
by interventions research and increase the level of theoretical
knowledge related to different interventions [5].
In this study, two types of research that compares
implementations, interventions and nursing cares, are referred
to as efficacy and effectiveness research. They are both classified
as intervention-based research that determine the impact of
an intervention [6]. The two fundamental words of efficacy and
effectiveness are different and should not be used mistakenly
instead of each other [6]. Therefore, the research question
determines the type of efficacy or effectiveness studies [2]. In terms
of research design, to determine the impact of an intervention
descriptive, correlation or observational designs, and to test the
efficacy or effectiveness of an intervention the experimental or
semi-experimental research designs are appropriate, respectively
[6]. And the most used designs are clinical trials [6,7], which are
widely welcomed by researchers, policy makers, stakeholders
and journals in comparing clinical approaches. In the meantime,
comparative effectiveness research and cost-effectiveness research
provide opportunities to make change in health care delivery and
improve patient outcomes [8]. Therefore, they can be considered
as national health policies and provide the opportunity to identify
the best evidence-based interventions which works in their target
communities and settings [9]. Because the lack of comparative
clinical data on effectiveness, costs of treatment, and care delivery
models can be a major challenge in optimizing health care value
[8]. The purpose of this study was to determine a new vision to
efficacy, comparative effectiveness and cost-effectiveness research
in nursing by an integrative review and Delphi technique in
combination.
Method
This study was conducted in two phases, an integrated literature review and a Delphi technique to confirm or emphasize on the collected data in the review.
Phase One – Integrative Review
The framework of Whittemore and Knafl [2005] for integrated
reviews was used to conduct the review [10]. This approach
allows the review to contain diverse primary research methods,
thus provides a comprehensive review needed for the topic of
interest [10]. The Science Direct, Pub Med and Google Scholar
databases were searched from 2000-2021 to identify the primary
papers. The PubMed search strategy was ((((“Comparative
Effectiveness Research”[Mesh]) OR “Cost-Benefit Analysis”[Mesh])
OR [“Treatment Outcome”[Mesh] OR “Therapeutic
Equivalency”[Mesh])) AND “Nursing”[Mesh]). In other databases
a full search strategy using the following terms were conducted:
effectiveness research/study, comparative effectiveness research/
study, cost-effectiveness research/study, efficacy research/study
and nursing/nurse led. Inclusion criteria consisted of any research
or paper in the nursing field without methodological limitations
about efficacy, comparative effectiveness, and cost-effectiveness
research, with full text access in English or Persian. To access
more papers and keep the maximum amount of data available, the
publication time limit was not considered. Critical Appraisal Skills
Program checklists were used to assess the quality of included
papers [11] for their trustworthiness, relevance and results [12].
Papers were excluded if they were conducted using efficacy or
effectiveness methods or a review of studies conducted with these
methods, did not explicitly mention their use of these methods,
and if they did not address the features of these types of research.
The initial search result included 17096 papers and 6 additional
records from other sources. Sixty-seven duplicate papers were
removed. One of the authors screened the title and abstract of
remaining 17033 papers based on inclusion criteria and excluded
16460 papers as irrelevant, abstract only, and inadequate quality. A
review full text for eligibility resulted in 548 more excluded papers
and 3 papers that were removed following critical appraisal. The
outcomes were discussed with other authors at any stage and
there was a full agreement with no conflict between them. Finally,
24 papers met the eligibility criteria to include in the integrative
review that all were published in English (Figure 1).
Figure 1: The most feasible position of fetoscope.
Phase Two – Delphi Technique
As review conducted, a summary of the interested research definitions was prepared and a questionnaire consisting of 10 questions was designed to ask nursing experts about efficacy and effectiveness studies using Delphi technique. The Delphi technique was used to collect a variety of data from nursing experts who were distanced and dispersed throughout Iran, whose perspective to research was very important due to their familiarity with a variety of research methodologies used in nursing studies. Nursing experts including editors in chief of nursing journals and research assistants of their nursing schools were the research population. The list of “Medical Journal Commission” of Iran was used to obtain the nursing journals which resulted in 31 journals. The email addresses of the 3 journals were invalid, so 28 journals were listed. The email address of the editors and research assistants were also searched separately. Finally, 27 editors in chief and 21 research assistants were directly enrolled in the study. Based on snowball sampling method, two international nursing experts with high citation rates who were introduced by one of the participants were included, too. Because the number of nursing experts was limited, all individuals in the research population including a total number of 50 people enrolled in the study. The questionnaire was sent to the journals in two occasions and once directly to editors in chief, the research assistants and two international nursing experts, respectively. The results of Delphi technique were used to confirm or emphasize on extracted data from integrated review of literatures.
Results
Phase one: Integrative Review
A total of 24 papers were enrolled in the review including articles, editorials and reports, published in English between 2000 to 2021. The Federal Coordinating Council [FCC] [13] and Institute of Medicine of The National Academies [IOM] [14] reports both were published in 2009. Also, the American Recovery and Reinvestment Act [ARRA] of 2009 and the patient protection and Affordable Care Act [ACA] of 2010 have increased the considerations to comparative effectiveness type studies widely [15], Which resulted in more research conducted in effectiveness scope since 2009. In this integrative review findings are presented based on the research area, objectives, application, significant features, appropriate/ common research design, and the study classification in two tables. (Table 1) [ 6-7,16-18] indicates a comparison of efficacy and effectiveness studies. (Table 1) [ 6-7,16-18] Also considering two types of studies in effectiveness research scope that can play an important role in determining health policies at national levels, (Table 2) [6-9,13-14,19-32] presents comparative effectiveness and cost-effectiveness studies separately.
Table 1: A comparison of Efficacy and Effectiveness Studies.
Table 2: Comparative Effectiveness and Cost-Effectiveness Studies.
Phase Two: Delphi Study
A total of 7 nursing experts [14%] participated in this study, including 5 editors in chief, 1 managing editor and 1 executive manager. In spite of follow-ups, other experts did not reply to the e-mails which was justified by the coincidence of data collection period with the second semester’s examinations and the limited time of study duration. Therefore, outcomes of the conducted single round Delphi, which resulted in mindful comments and answers of participated experts, was used to confirm or emphasize on extracted data from integrated review of literatures. The Delphi phase data are classified in 6 finding areas about Efficacy, Comparative Effectiveness and Cost-Effectiveness research.
Knowledge of Experts About Efficacy, Comparative Effectiveness and Cost-Effectiveness Research
The methodology of these three studies is not mentioned explicitly in the curriculum and are not taught in detail, which results in poor knowledge and awareness of students, faculty members and nursing experts about differences and the nature of these studies (Participants No. 6 and 7). “… This lack of knowledge represents a training deficiency for nurses, journals and even the teachers …” (P.No. 6). On the other hand, the general information of individuals indicates that all three types of research usually take quantitative methods (P.No. 5) and are often conducted as a trial which assesses an impact, so differentiation between them may be difficult ([P.No. 2). Therefore, if the type of study was not explicitly mentioned, regard to the title or inappropriate use of vocabulary due to assessing the impact of an intervention or care, it may mistakenly be considered as effectiveness and usually comparative form of studies, while the underlying nature of the study is efficacy (P.No. 6 and 7).
Perspectives of Experts on Efficiency Studies, Comparative Effectiveness and Cost-Effectiveness Research
Considering the fact that nursing is an evidence-based discipline, and all of these three studies play an important role in evidence production, there should be a focus on the studies providing opportunities to produce accurate evidence in a comparative effectiveness or cost-effectiveness manner (P.No. 1, 2, 3 and 6). These studies possess high importance and offer applicable solutions as appreciated and valuable evidence that is capable of implementing an intervention with comparative accuracy and lower cost (P.No. 2 and 4).
Ideology of Experts About Nursing Research Priorities
Demonstrating the cost-effectiveness or cost-effectiveness of nursing, especially registered nursing is an imperative (P.No. 5, 6 and 7). Efficacy, comparative effectiveness, cost-effectiveness, or cost benefit of different implementations, interventions and nursing cares are appropriate priorities for nursing research (P.No. 1). Also, nursing beyond the acute care seems necessary (P.No. 5). Therefore, a wide range of actions can be mentioned for research conductions. For example, music therapy, heat/cold therapy, mental imagery, massage, etc. and comparing modern and traditional practices are notable (P.No. 1 and 2).
Perspectives of Experts on Obstacles and Strategies in Conducting Effectiveness Research
Effectiveness studies have considered throughout the world alongside efficacy studies (P.No. 5), but in Iran, despite the wide range of feasible research methods, yet they are not considered enough (P.No. 2), Which results due to the existing obstacles leading to obscurity and limited conducting of effectiveness studies, or even insufficient quality of conducted studies (P.No. 1). The mentioned obstacles, included lack of participation or inadequate power of nurses in healthcare policymaking (P.No. 3 and 6), inter and intradisciplinary conflicts or lack of cooperation (P.No. 1), unknown influence of journals and editors on research (P.No. 6), research criticizing efficaciously by stringent controls (P.No. 6 and 7), lack of research outcomes influencing the practice (P.No. 1), staff shortage either faculty members or clinical nurses (P.No. 3)and financial and time limitations (P.No. 1). “… In many medical universities, the lack of appropriate funding and, more importantly, the lack of time lead to less tendencies to conduct these types of studies timeconsuming studies. … Faculty members are busy with educational duties, so they do not have enough time to conduct these studies or because of the academic years increase in postgraduate students, such studies are not welcomed by colleagues or students. (P.No. 1).” The suggested strategies to reduce the existing obstacles, included providing clinical nursing specialties, Doctor of Nursing practice [DNP] and nurse practitioners, to achieve more power and position in healthcare (P.No. 2), making interests on studying health management alongside clinical services (P.No. 3) and providing information and support among colleagues (P.No. 1 and 4),
Ideology of Experts about Motivating Factors to Conduct Effectiveness Studies
In order to motivate nursing researchers to conduct effectiveness studies, it was recommended as necessary to increase the level of professional independence in nursing and raise the interdisciplinary coordination and partnership, which requires time and effort for nursing to reach its main position (P.No. 1 and 2). Other mentioned motivators included providing enough time and budget (P.No. 1), receiving support and information from colleagues (P.No. 1 and 4), more training in health economics and better collaboration with health economists [P.No. 5 and 7], and leading research projects and theses to effectiveness methodologies by nursing professors to make the administration context managerially (P.No. 3). “I think these studies would be suitable for PhD level study not undergraduate and masters as they will take a long time and require expertise and training not available at undergraduate level (P.No. 5).” The importance of effectiveness research could be identified by the implementation of the theses (P.No. 1-7). Also training workshops (P.No. 4) and qualitative methods (P.No. 2), emphasized on theses conductions. Despite this importance, research projects with more feasibility and less time limitations could make better research outcomes in effectiveness scope (P.No. 3).
The Role and Importance of Effectiveness Studies in Nursing Professional Development
Initially nursing professional development needs to be studied more and deeply (P.No. 4). In spite of this necessity, conducting effectiveness studies, can lead to nursing professional development if their outcomes are used in practice to solve nursing care problems (P.No. 1). These projects improve the quality of nursing care, which is the main fundamental feature of service providing professions such as nursing (P.No. 3, 6 and 7). Otherwise conducting effectiveness studies because of their possible distinctions or compulsory being, without interest and lack of adequate regulatory standard monitoring systems, reduces the motivation to use the results in practice (P.No. 1), Which is modified by ethical considerations, creation of advisory groups in health centres for researchers and assessment of clinical trials conducting process that can increase the quality of studies (P.No. 1). On the other hand, it should be noticed that effectiveness may be well represented, but there is a threat about weak point of alternative models which tends to need further exploration and search despite the proven positive results (P.No. 5).
Conducting such studies can express the profession, especially if they are published in valid journals (P.No. 2). Definitely there is no limitation for effective studies publication if they are conducted in a high quality with powerful research designs and well reported (P.No. 1-7). Among these three types of research, it seems that comparative effectiveness is preferable because of tangible evidence production, but the quality of study determines the publication priorities than methodology itself (P.No. 1-4). Publishing these types of strong studies in Iranian nursing journals can validate them, too (P.No. 4). Therefore, conducting more powerful studies by nursing professors can provide an opportunity to improve clinical conditions and create science production settings over time (P.No. 6 and 7).
Discussion
This study was conducted to determine a new vision to efficacy,
comparative effectiveness and cost-effectiveness research in
nursing. For this purpose, an integrative review in combination with
Delphi technique was conducted. The emphasized and important
role of efficacy and effectiveness research in producing clinical
evidence [6,13,14,16,17], mentioned by nursing experts, considers
the evidence-based nursing values. So, it is an imperative to transfer
knowledge and evidence to practice [4], which results in leading
the research to real practice settings as clinical outcomes research.
Other possible results include making strong and reliable evidence,
conducting any type of research considering the needed priorities
for each nation based on health policies and validating the existing
body of knowledge or making new knowledge infrastructures.
The integrative review indicated that evidence made by efficacy;
comparative effectiveness and cost-effectiveness research are
complementary to each other, so more training workshops are
needed to make theoretical sensibilities in nursing research. The
efficacious intervention in an Ideal, selective and controlled setting
[6,16,17] should be translated to a real-world setting to prove their
effectiveness [6,13,14,16,17] in comparison to the routine care
[8,13,14,16,31] in real community populations [13,14,16] with the
least possible exclusion criteria [17]. This is a research process to
confirm evidence and make them applicable in nursing practice as
evidence-based nursing cares.
The patient-centred evidence resulted from comparative
effectiveness research [3,16,21,22,25] can increase the cost, because
it considers the effectiveness independently without noticing its
costs [19]. While increased costs of healthcare services which
results in financial burden for both patients and their families and
the government itself, makes the healthcare policies to determine
effective evidence with lower costs; that is also mentioned by nursing
experts. So, using cost-effectiveness analysis is indeed to reduce
the healthcare costs [7,19,31]. But the usefulness of comparative
effectiveness research is agreed [15], and in serious diseases even
cost-effectiveness research recommend the effective intervention
without considering its costs while this is vice versa in less serious
diseases and costs of intervention is prior to their effectiveness
[19]. So, everything about making clinical evidence depends on the
purpose of researcher and existing policies, also efforts on nursing
professional development with increased influencing power
on healthcare policies are noticeable. Nursing profession as an
independent discipline needs its own strong body of knowledge. As
expectations increase for comparative effectiveness research, more
gaps appear in body of knowledge and research infrastructures that
should be addressed in order to achieve informed decisions and
better outcomes [14]. But despite the increased use of this patientcentred
research method, controlled trials are more published [33]
while the clinical outcome research can provide a stronger body of
knowledge in nursing [2,6]. Also nursing experts claimed that there
are a lot of gaps in nursing, but the existing obstacles fetter these
studies. In order to modify the research obstacles, it is necessary to
introduce methodologies in detail, which will motivate the faculty
members, researchers and students conduct more patient-centred
research to make strong body of knowledge for nursing. Therefore,
there is a need for standard rules and monitoring systems to
conduct high quality studies. Moreover, the effort to strengthen the
nursing profession tends to take part in healthcare policy making.
Generally determining that which evidence-based intervention,
in whom and where is more effective [9], needs more patient
involvement in the study [21], according to the community drive,
healthcare policies and stakeholders’ penetration [9]. And it totally
is dependent to the purpose of knowledge translation to practice,
because inter and intra-disciplinary collaborations guarantee the
research support, funding and evidence-based nursing practice.
Yet the least information of nursing experts about the existing
differences between efficacy, comparative effectiveness and costeffectiveness
research is a challenging issue. And it is still possible
to use these two words mistakenly [6]. Therefore, providing
information especially for faculty members and nursing journals is
so imperative.
Limitations
The study limitations include lack of access to some databases, limited access to full text of all papers, failure to achieve sufficient with methodological-focused approaches especially in efficacy scope and emerging effectiveness studies in health and nursing in recent decade since 2009. In order to reduce the impact of limitations abstract only papers were reviewed, to prepare their full text if they were appropriate to be enrolled in the integrative review, but none met the eligibility criteria. Instead, the editorials and reports which emphasized on methodologies enrolled to the review.
Conclusion
The efficacy and effectiveness research are important in producing complementary evidence to improve patient care outcomes. Recently, due to the need for comprehensive and accurate information to be more applicable in clinical practice, comparative effectiveness research is more considered and seems to be more important across the world. The effectiveness outcomes origin in the real clinical practice settings and can be extracted using diverse study designs depended on research question and aims. Therefore, the efficacious implementations or interventions should be studied for their effectiveness in real-world settings to become applicable. The obtained evidence needs to be reviewed to make a body of knowledge. The health policies should provide opportunities to test the new knowledge in implementation research in practice. The main conclusion of completing this research process leads to reduce the gap between theory and practice in nursing and its professional development. In spite of the increased consideration to effectiveness research, both in terms of comparative effectiveness and cost-effectiveness, still efficacy studies are more welcomed than effectiveness studies in Iran which demonstrates a controlled impact of an intervention in an ideal setting which is not necessarily applicable in real-world settings. But according to the economic constraints and the high costs of health services, cost-effectiveness studies are increasingly considered in Iran; because costeffectiveness research, determines both costs and effectiveness of two or more interventions correspondingly.
Funding/Support
Tabriz University of Medical Sciences, Tabriz, Iran provided facilities and supported this project.
Acknowledgment
We would like to acknowledge the nursing experts who participated in the study and the journals staff who referred the e-mails to their editors in chief in our follow ups.
Conflict of Interests
None of the authors has any conflict of interest to declare.
References
- Bryant-Lukosius D, DiCenso A, Browne G, Pinelli J (2004) Advanced practice nursing roles: Development, implementation, and evaluation. J Adv Nurs 48(5): 519-29.
- Goss CH, Tefft N (2013) Comparative effectiveness research - What is it and how does one do it? Paediatr Respir Rev 14(3): 152-6.
- Tunis SR, Clarke M, Gorst SL, Gargon E, Blazeby JM, et al. (2016) Improving the relevance and consistency of outcomes in comparative effectiveness research. J Comp Eff Res 5(2): 193-205.
- Hickman LD, DiGiacomo M, Phillips J, Rao A, Newton PJ, et al. (2018) Improving evidence-based practice in postgraduate nursing programs: A systematic review: Bridging the evidence practice gap (BRIDGE project). Nurse Educ Today 63: 69-75.
- Forbes A (2009) Clinical intervention research in nursing. Int J Nurs Stud 46(4): 557-68.
- Grove SK, Burns N, Gray JR (2013) The Practice of Nursing Research: Appraisal, Synthesis, and Generation of Evidence (7th)., Elsevier Saunders, pp. 755.
- Noyes K, Holloway RG (2004) Evidence from Cost-Effectiveness Research. NeuroRx 1(3): 348-55.
- Stone PW, Cohen C, Pincus HA (2017) Comparative and cost-effectiveness research: Competencies, opportunities, and training for nurse scientists. Nurs Outlook 65(6): 711-7.
- Price Haywood EG (2015) Clinical Comparative Effectiveness Research Through the Lens of Healthcare Decisionmakers. Ochsner J 15(2): 154-61.
- Whittemore R, Knafl K (2005) The integrative review: Updated methodology. J Adv Nurs 52(5): 546-53.
- (2018) CASP UK CASP Checklists - CASP - Critical Appraisal Skills Programme.
- (2018) Casp UK CASP - Critical Appraisal Skills Programme.
- (2009) Federal Coordinating Council. Report to the President and the Congress on Comparative Effectiveness Research. Washengton, DC: U.S. Department of Health and Human Services.
- Institute of Medicine of The National Academies. Initial National Priorities for Comparative Effectiveness Research [Internet]. National Academy of Sciences. Washington, DC: THE NATIONAL ACADEMIES PRESS 2009, pp. 253.
- (2012) Health Biomedical Science & Society Initiative. Moving Forward with Comparative Effectiveness Research. Aspen Institute. Health Biomedical Science & Society Initiative.
- Kim SY (2013) Efficacy versus effectiveness. Korean J Fam Med 34(4): 227.
- Pagoto SL, Lemon SC (2013) Efficacy vs effectiveness. JAMA Intern Med 173(13): 1262-3.
- Kirpalani H, Truog WE, D’Angio CT, Cotten M (2016) Recent controversies on comparative effectiveness research investigations: Challenges, opportunities, and pitfalls. Semin Perinatol 40(6): 341-7.
- Shafrin J (2010) Comparative Effectiveness vs. Cost Effectiveness Research. Healthcare Economist 2.
- Devine EB, Perfetto E, Pickard AS, Schumock GT, Segal B, et al. (2018) Nine years of comparative effectiveness research education and training: initiative supported by the PhRMA Foundation. J Comp Eff Res 7(2): 167-75.
- Mullins CD, Abdulhalim AM, Lavallee DC (2012) Continuous Patient Engagement in Comparative Effectiveness Research. JAMA 307(15): 1587-8.
- Knottnerus JA, Tugwell P (2018) Comparative effectiveness research requires competitive effectiveness. J Clin Epidemiol 94: v-vi.
- Safdar NM, Siegel E, Erickson BJ, Nagy P (2011) Enabling Comparative Effectiveness Research with Informatics. Acad Radiol 18(9): 1072-6.
- Howie L, Hirsch B, Locklear T, Abernethy AP (2014) Assessing the Value of Patient- Generated Data to Comparative Effectiveness Research. Patients Res 33(7): 1220-8.
- Basch E, Aronson N, Berg A, Flum D, Gabriel S, et al. (2012) Methodological standards and patient-centeredness in comparative effectiveness research: The PCORI perspective. JAMA 307(15): 1636-40.
- Patel I, Rarus R, Tan X, Lee E, Guy J, et al. (2015) Investigation of comparative effectiveness research in Asia, Europe, and North America. Indian J Pharmacol 47(6): 585.
- Naik AD, Petersen LA (2009) The Neglected Purpose of Comparative-Effectiveness. N Engl J Med 360(19): 1929-31.
- Sox HC, Goodman SN (2012) The Methods of Comparative Effectiveness Research. Annu Rev Public Health 33(1): 425-45.
- Conn VS, Ruppar TM, Phillips LJ, Chase JD (2012) Using meta-analyses for comparative effectiveness research. Nurs Outlook 60(4): 182-90.
- Atkins D, Kupersmith J (2010) Implementation Research: A Critical Component of Realizing the Benefits of Comparative Effectiveness Research. Am J Med 123(12): e38-45.
- Bensink ME, Eaton LH, Morrison ML, Cook WA, Curtis RR, et al. (2013) Cost effectiveness analysis for nursing research. Nurs Res 62(4): 279-85.
- Gray EA, Thorpe JH (2015) Comparative effectiveness research and big data: Balancing potential with legal and ethical considerations. J Comp Eff Res 4(1): 61-74.
- Hester LL, Poole C, Suarez EA, Der JS, Anderson OG, et al. (2017) Publication of comparative effectiveness research has not increased in high-impact medical journals, 2004-2013. J Clin Epidemiol 84: 185-7.
