info@biomedres.us   +1 (720) 414-3554
  One Westbrook Corporate Center, Suite 300, Westchester, IL 60154, USA

Biomedical Journal of Scientific & Technical Research

May, 2022, Volume 43, 4, pp 34725-34736

Research Article

Research Article

Analysis of Spins in Randomized Controlled Trials of Topical Agents on Rosacea

Hongzuo Guo1,2#, Jing Zeng1#, Qixin Han3, Shuai Qiao1 and Yan Wu1*

Author Affiliations

1Department of Dermatology, The First Hospital of China Medical University, China

2Department of Dermatology (Medical Cosmetology), Artemis Aesthetic Medical, China

3Department of Dermatology, Taiyuan Central Hospital of Shanxi Medical University, China

#These authors contributed equally to this manuscript.

Received: April 25, 2022 | Published: May 02, 2022

Corresponding author: Yan Wu, Department of Dermatology, The First Hospital of China Medical University, Shenyang, 110001, China

DOI: 10.26717/BJSTR.2022.43.006924

ABSTRACT

Background: Under certain motives, authors may subconsciously or consciously cause the appearance of spins which defined as the use of specific reporting or the manipulation of language to highlight the beneficial effect of the treatment in terms of efficacy or safety.

Aims: To appraise potential spins in Randomized Controlled Trials (RCTs) that applied FDA approved topical agents to rosacea.

Methods: A systematic review of the literature was performed to identify vehicleor placebo-controlled RCTs of 5 topical agents including metronidazole, azelaic acid, sulfacetamide/sulfur, brimonidine, ivermectin on rosacea. Spins of each included RCT were evaluated, and the descriptions of spins were conducted in abstract section and in main text section, respectively. Besides, the severity of spin in RCTs with nonsignificant primary outcomes as well as the relationship between spin and journal impact factor were also assessed.

Results: A total of 20 RCTs met the inclusion criteria and were included in this study. In abstract section, the spin of “inappropriate extrapolation of results” was the most common one (in 20% RCTs) comparing to other two items. In main text section, the spin of “using multiple primary outcomes” presented in 100% RCTs, and “focusing on within-group comparisons” and “reporting percent changing from baseline” happened in 40% and 30% RCTs, respectively. Five studies had statistically nonsignificant results for primary outcomes and were evaluated spin severities, and 4 RCTs had high level of spins. When IF=0, the spins appeared in 20% abstract of RCTs and 100% main text. Unexpectedly, 100% RCTs published in 0< IF< 2 journals showed spins in both abstract and main text.

Conclusion: The spins involved in the aspect of “Inappropriate statistical analysis” in main text, especially “using multiple primary outcomes”, were commonly presented in RCTs. The “claiming an effect for statistically nonsignificant results”, “claiming an effect for no statistically analyzed results” and “Inappropriate extrapolating results from specific outcome to global improvement” were also relatively common.,4 out of the 5 RCTs with statistically nonsignificant results for primary outcomes had high level of spins. Moreover, the severity of spin seemed to be independent of the impact factor.

Keywords: Rosacea; Spin; RCT; Topical

Introduction

Rosacea is a chronic facial skin disease and occurs most commonly in adults aged 30 to 50 years [1]. The disease is characterized by marked involvement of the central face with transient or persistent erythema, inflammatory papules or pustules, telangiectasia, or hyperplasia. The subjective symptoms may be stinging, burning, and/or flushing. Since the unaesthetic appearance and annoying symptoms affect patients’ life quality to a certain extent, patients may feel ashamed, embarrassed and anxious, and even suffer from lack of self-confidence, inferiority and depression [2]. Randomized controlled trial (RCT) is regarded as a gold standard in therapeutic research and fundamental to the evidence-based adoption of clinical interventions. To guarantee objectivity, transparency, and applicability, rigorous guidelines have been developed concerning of planning, implementation, and reporting of RCTs, by enactments of initiatives such as consolidated standards of reporting trials (CONSORT) and mandatory prepublication trial registration. It means that reporting results are basically in accordance with a prespecified protocol, including outlines of intervention groups, primary and secondary endpoints, and statistical analyses [3]. Since RCTs are important for clinical practice, it should be reported properly. Nevertheless, under certain motives, authors may subconsciously or consciously shape the impression of their results for readers when reporting their RCTs and thus spins will appear. Spin is defined as the use of specific reporting or the manipulation of language to highlight the beneficial effect of the treatment in terms of efficacy or safety [3]. Spin overstates efficacy and/or understates harm, and thus potentially misleads readers from the likely truth of results. Spin may lead to inaccurate estimation of the benefit of interventions and affect patient care [4-6]. Topical agents are relatively safe treatment for skin disorders like rosacea and can target local lesions directly. The U.S. Food and Drug Administration (FDA) has approved five topical agents, i.e., metronidazole, azelaic acid, sulfacetamide/sulfur, brimonidine, and ivermectin to treat rosacea [7]. To our knowledge, no study has been conducted to evaluate spins of RCTs with these topical agents treating rosacea. The purpose of this study is to evaluate potential spins in RCTs that applied FDA approved topical agents to rosacea.

Methods

Selection of Studies

Systematic searches of scientific literature in 3 English databases including PubMed, the Cochrane Library and EMBASE, which were conducted from inception to March 2021. Randomized controlled trials (RCTs) on rosacea with 5 FDA approved topical agents (metronidazole, azelaic acid, sulfacetamide/sulfur, brimonidine, ivermectin) were searched. The following searching terms were used: (metronidazole [Title/Abstract] OR azelaic acid [Title/Abstract] OR nonanedioic acid [Title/Abstract] OR sulfacetamide [Title/Abstract] OR sulfur [Title/Abstract] brimonidine [Title/Abstract] OR ivermectin [Title/Abstract]) AND rosacea [Title/Abstract] AND (placebo) [Title/Abstract] OR vehicle [Title/Abstract]). After duplicates being removed, two investigators (Hongzuo Guo, Jing Zeng) independently reviewed all the titles and abstracts to include potential studies. Finally, the full texts of remaining studies were further reviewed to determine whether these articles could be included.

Inclusion and Exclusion Criteria

Inclusion criteria:

a) RCTs,

b) Vehicle- or placebo-controlled studies,

c) Studies written in English or Chinese,

d) Studies using one or more of the 5 FDA approved topical agents (single or combined) on rosacea.

Exclusion Criteria:

1. Not human studies

2. Studies using non-topical interventions or non-FDA approved topical agents.

Evaluation of Spins

Spins of each included RCT were evaluated on the basis of previously established strategies [7-10]. The descriptions of spins were conducted in abstract section (3 Items, AItem1- AItem3) and in main text section (17 Items, MItem1-MItem13, MItem14*-MItem17*), respectively. And for main text, the 17 Items were under 3 main aspects, i.e., inappropriate statistical analysis (MItem1-MItem6), inappropriate interpretation of results (MItem7-MItem10), and inappropriate extrapolation of results (MItem11-MItem13, MItem14*-MItem17*). The detailed Items were presented in Figure 1. According to Dr. Yavchitz, it is necessary to develop a unified system to rate the severity of spin in articles [8]. In an exploratory evaluation system proposed by Boutron and colleagues [10], when RCTs with statistically nonsignificant results for primary outcomes (MItem13*-MItem17* in our studies), the degrees of spin were classified as high level, moderate level, or low level. The evaluation flow chart was mapped in Figure 2, based on the existence of MItem13*-MItem17* or not. All spin Items of each included study were evaluated by 2 reviewers (Hongzuo Guo, Jing Zeng). Disagreements were resolved by a third investigator (Yan Wu) via discussion and consensus.

Note: *Represents spin in RCT which had statistically nonsignificant results for primary outcomes.

Figure 1: Survey used to evaluate strategies of spin.

Note: Mlteml4, Recommend to use the treatment in clinical practice; MItem15, Not acknowledge of the statistically nonsignificant results for the primary outcome; MItem16, Not recommend for further trials; MIteml7; Not report with uncertainty in the framing; High, the spin level of RCT is high; Moderate, the spin level of RCT is moderate; Low, the spin level of RCT is low.

Figure 2: Flow chart evaluating the level of spin of RCTs with nonsignificant outcome.

Results

A total of 20 RCTs met the inclusion criteria and were included in the study (Figure 3) [11-30]. The information of intervention and outcome measurement of each RCT were summarized in Table 1. All spin items were assigned to each RCT to identify potential spins and an example of each spin item was shown in Table 2 to make it more understandable. In brief, 8 RCTs [14-16,19,27-30] had spins in both abstract section and main text section, and 12 RCTs [11- 13,17,18,20-26] had only main text spins. The detailed results of spins identification were shown in Figure 1 and Table 2.

Figure 3: Flow chart depicting study selection for systematic review.

Table 1: Summary of intervention and outcome measurement of included studies.

Note: NA, not available; IGA, investigator global assessment; PGA, patient global assessment; CEA, clinician erythema assessment; AEs, adverse events; PRO, patient-reported outcomes; ILC, inflammatory lesion count; QoL, quality of life; DLQI, dermatology life quality index; RosaQoL, a rosacea-specific instrument; SSR, symptom severity rating; EQ-5D-5L, Euro QOL 5-dimension 5-level questionnaire; PGE, physician’s global evaluation; SGE, subject’s global evaluation; PSA, patient’s self-assessment; PAA, patient’s assessment of appearance; FRQ, a facial redness questionnaire.

Table 2: Spins in each included study.

Note: AItem1, focusing selectively on statistically significant efficacy outcomes; AItem2, reporting improvement but ignore control group; AItem3, inappropriate extrapolation of results; MItem1, focusing on within-group comparisons; MItem2, reporting results selectively; MItem3, reporting percent changing from baseline; MItem4, relying on single subject to claim results of entire sample; MItem5, using multiple primary outcomes; MItem6, using multiple statistical analyses on same data; MItem7, focusing on interim analyses to give more weight to nonsignificant findings; MItem8, claiming an effect for statistically nonsignificant results; MItem9, claiming an effect for no statistically analyzed results; MItem10, claiming an effect for only reporting one of relative effect or absolute effect; MItem11, extrapolating the findings to a different population or setting; MItem12, extrapolating the findings to a different intervention; MItem13, extrapolating inappropriately results from specific outcome to global improvement; MC14*: Recommend to use the treatment in clinical practice; MC15*: Not acknowledge of the statistically nonsignificant results for the primary outcome; MC16*: Not recommend for further trials; MC17*: Not report with uncertainty in the framing.

Spins in Abstract

No RCT had all the 3 spin AItems. Most RCTs (60%) had no spins in abstract [11-13,17,18,20-26]. Each of the 40% RCTs which had AItems had 1 AItem. Respective 5%, 10%, and 20% RCTs presented AItem1, AItem2, and AItem3.

Spins in Main Text

All RCTs had at least 1 MItems. Of all the 13 MItems, 5 RCTs [12,14-16,19] presented more than 4 MItems, 12 RCTs [11,17,19- 25,27-30] had 2-3 MItems, and 3 RCTs [13,18,26] had only 1 MItem. In the aspect of “Inappropriate statistical analysis”, all included RCTs had MItem5. In contrast, none of the RCTs had MItem2, MIem4 or MItem6. Respective 40% RCTs [12,15-17,21,23-24,27] and 30% RCTs [12,15-17,22,29] had MItem1 and MItem3. In the aspect of “Inappropriate interpretation of results”, 5% RCTs [12], 25% RCTs [12,14,19,20,29], 25% RCTs [17,21,22,25,29] and 10% RCTs [19,28] had spins of MItem7, MItem8, MItem9 and MItem10, respectively. With regard to the 3 spins of “inappropriate extrapolation of results” in conclusion section, no RCT presented MItem12, but 20% RCTs had MItem13 [14,16,20,28], and 15% RCTs showed MItem11 [14,23,30]. When RCTs with statistically nonsignificant results for primary outcomes (n=5), the latter 4 spins in conclusion were evaluated. All of the 5 studies had MItem15*, and 80% of the 5 studies had MItem16* and MItem17*, respectively. About 40% RCTs had MItem14*.

The Severity of Spin

The statistically nonsignificant results for primary outcomes present in 5 RCTs and the severity of spin was evaluated of each RCT. High level of spins appeared in 4 RCTs [13-15,22] and low level of spins appeared in 1 RCTs [25], respectively. However, no RCTs appeared moderate level of spin.

The Relationship between Journal Impact Factor and Spin

We rated the impact factors (IF) of included journals by 4 scales: 0, 0< IF < 2, 2 ≤IF< 4, 4 < IF. When IF=0, the spins appeared in 20% abstract of RCTs and 100% main text. Unexpectedly, the RCTs published in 0 < IF <2 journals showed highest incidences (100%) of spins in both abstract and main text. When the journals with 2≤IF<4, about 66.67% RCTs had spins in abstract and 100% in main text. Only 1 journal with IF>4 showed spins in main text rather than in abstract. Figure 4 presented spin incidence rates in different journals classified by IF.

Figure 4: Spin incidence rates in different journals classified by impact factor.

Potential Motives of Spins

Assessing disclosures, sources of funding, or authors ‘affiliations with manufacturers of the tested products to appraise conflicts of interest. Sources of funding were reported in 16 RCTs [11,14-18,19- 23,26-30], and 14 RCTs [11,14-17,19-23,30] were supported by companies. Of all the 20 RCTs, 16 RCTs reported disclosures [11,14- 23,26-30]. 10 RCTs reported that authors and the tested products had affiliated to the same companies [11,14,19-22,26-28,30].

Discussion

The current systematic review showed that the spin of “using multiple primary outcomes” was presented in 100% RCTs. In a previous review about analysis of spin in the studies of topical treatments for photoaged skin [7], this kind of spin was also the most common one and happened in 95% included studies. In our result, the secondary commonly happened spin was “focusing on within-group comparison” in 40% RCTs, while it had a higher prevalence of 75% in a previous review [7]. And “reporting percent changing from baseline” was the third spin presented in 30% RCTs, which was similar to a previous report [7]. Notably, all above three spins belong to “Inappropriate statistical analysis” aspect of main text. It reminds research to pay more attention to avoiding these problems, especially “using multiple primary outcomes”, when designing and implementing a RCT.

In our results, the spins of “claiming an effect for statistically nonsignificant results” and “claiming an effect for no statistically analyzed results” also had a relatively high prevalence of 25% comparing to other items, and both belonged to the aspect of “Inappropriate interpretation of results” in main text. These two spins had a medium prevalence of 50%-60% in a previous report [7]. In another previous review about evaluating spin in wound care research [31], the spin “claiming an effect for statistically nonsignificant results” present the highest prevalence. In addition, the spin of “Inappropriate extrapolating results from specific outcome to global improvement” showed the highest prevalence of 95% in a previous review [7]. In our review, this spin in abstract part also presented the highest prevalence of 20% comparing to the other two items in abstract. And in main text part, it also presented a relatively higher prevalence of 20% comparing to other items. These results suggested that “Inappropriate interpretation of results” and “Inappropriate extrapolation of results” should also be taken into consideration even when “Inappropriate statistical analysis” was avoided.

RCTs with statistically significant results for outcomes are published more often and more rapidly than those with statistically nonsignificant results [32]. However, statistically nonsignificant results for primary outcomes are also of great values, as the negative results can remind other researchers to avoid wasting time or investment in these fields. According to Boutron and colleagues’s report, when RCTs had statistically nonsignificant results for primary outcomes, 4 spin items (MItem14*-MItem17*) of “Inappropriate extrapolation of results” were suggested to be evaluated [10]. These items seemed to be rather important because the spin severity can be assessed based on them [8]. Especially, “recommend using the treatment in clinical practice” was the most important one to judge whether a study had a high level of spin. Besides, the “not acknowledge of the statistically nonsignificant results for the primary outcome” was another key spin to decide whether a study can be directly classified into a type with low-level spin. We can judge the severity of spin of a RCT according to Figure 2, among the 5 included RCTs of our study, 80% had a high level of spin, none had medium one, and 20% had low one, respectively. When the RCTs with statistically nonsignificant results possess low level of spins, they should also be emphasized.

In our 4 scales of IF: 0, 0 < IF<2, 2≤IF<4, 4 < IF, the spins appeared in main text all were 100%, but the spins appeared in abstract was 20%, 100%, 66.67% and 0%, respectively. It indicated that the paper which was published in a journal with high impact factor may also have spins and the presence of spin seemed to be independent of the impact factor of the journal. If that is the case, the negative impact caused by spins in this paper will be more extensive. Since the evaluation of spin had a subjective element, it might not fully reflect the actual situation although two independent reviewers were involved in evaluation phase to minimize negative effects of such subjectivity. In the present study, a total of 14 RCTs [11,14- 17,19-23,26-28,30]. were supported by companies. Previous studies showed a positive relation between financial ties and favorable conclusions stated in trial reports [33]. Some pressures might influence the ways that data are collected, interpreted, and portrayed, such as external funding sources, potential for promotion, and potential for recognition [3,34]. Therefore, researchers should state their sources of funding and the affiliation with researchers [35,36].

A few ways should be raised to avoid the appearance of spin as much as possible. Firstly, selection of a recognized and unified standard as the outcome measurements of a disease can reduce the possibility of “use multiple primary outcomes” to some extent. Besides, writing scientifically and avoidance of incorrect motivation can effectively reduce occurrence rate of spin. When designing a RCT and dealing with data, professional statisticians should be consulted to avoid spins caused by improper statistic methods. What’s more, whether each paper should spare a certain length to evaluate the spin level of itself should arouse scientific researchers’ thinking. This systematic review had some limitations. Firstly, it might not include all strategies of potential spins. Secondly, the evaluation of spin was a subjective procedure which might not reflect a real condition. Thirdly, we cannot say to what extent the spins we identified might be deliberately misleading. Finally, all spin items were not ranked by severity. The severity of a spin depends on the evaluation criteria used by the authors.

Conclusion

In summary, this study has classified a scheme of spins to evaluate spins in RCTs of topical treatment for rosacea. The spins involved in the aspect of “Inappropriate statistical analysis” in main text, especially “using multiple primary outcomes”, were commonly presented in RCTs. The “claiming an effect for statistically nonsignificant results”, “claiming an effect for nonstatistically analyzed results” and “Inappropriate extrapolating results from specific outcome to global improvement” were also relatively common. Of the 5 RCTs with statistically nonsignificant results for primary outcomes, 80% RCTs had high level of spins. The presence of spin seemed to be independent of the impact factor of the journal. The presence of spin may potentially mislead readers from the likely truth of the results. Investigators, editors and peer reviewers need to be aware of the issue of spin and the importance of objectivity in research reporting. Researchers must apply appropriate application of statistical analytic methods and report data objectively. Additionally, if all users of research receive a critical appraisal skills training, it may be beneficial to combat the impact of spin on clinical decision making.

Disclosure Statement

The authors have no commercial associations that might create a conflict of interest in connection with submitted manuscripts.

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Data Availability Statement

All data generated or analysed during this study are included in this published article.

Authors Contribution Statement

Hongzuo Guo and Jing Zeng: Study design, literature research and manuscript writing; Qixin Han: Statistical analysis; Yan Wu and Shuai Qiao: Manuscript revision/review.

References

Research Article

Analysis of Spins in Randomized Controlled Trials of Topical Agents on Rosacea

Hongzuo Guo1,2#, Jing Zeng1#, Qixin Han3, Shuai Qiao1 and Yan Wu1*

Author Affiliations

1Department of Dermatology, The First Hospital of China Medical University, China

2Department of Dermatology (Medical Cosmetology), Artemis Aesthetic Medical, China

3Department of Dermatology, Taiyuan Central Hospital of Shanxi Medical University, China

#These authors contributed equally to this manuscript.

Received: April 25, 2022 | Published: May 02, 2022

Corresponding author: Yan Wu, Department of Dermatology, The First Hospital of China Medical University, Shenyang, 110001, China

DOI: 10.26717/BJSTR.2022.43.006924

ABSTRACT

Background: Under certain motives, authors may subconsciously or consciously cause the appearance of spins which defined as the use of specific reporting or the manipulation of language to highlight the beneficial effect of the treatment in terms of efficacy or safety.

Aims: To appraise potential spins in Randomized Controlled Trials (RCTs) that applied FDA approved topical agents to rosacea.

Methods: A systematic review of the literature was performed to identify vehicleor placebo-controlled RCTs of 5 topical agents including metronidazole, azelaic acid, sulfacetamide/sulfur, brimonidine, ivermectin on rosacea. Spins of each included RCT were evaluated, and the descriptions of spins were conducted in abstract section and in main text section, respectively. Besides, the severity of spin in RCTs with nonsignificant primary outcomes as well as the relationship between spin and journal impact factor were also assessed.

Results: A total of 20 RCTs met the inclusion criteria and were included in this study. In abstract section, the spin of “inappropriate extrapolation of results” was the most common one (in 20% RCTs) comparing to other two items. In main text section, the spin of “using multiple primary outcomes” presented in 100% RCTs, and “focusing on within-group comparisons” and “reporting percent changing from baseline” happened in 40% and 30% RCTs, respectively. Five studies had statistically nonsignificant results for primary outcomes and were evaluated spin severities, and 4 RCTs had high level of spins. When IF=0, the spins appeared in 20% abstract of RCTs and 100% main text. Unexpectedly, 100% RCTs published in 0< IF< 2 journals showed spins in both abstract and main text.

Conclusion: The spins involved in the aspect of “Inappropriate statistical analysis” in main text, especially “using multiple primary outcomes”, were commonly presented in RCTs. The “claiming an effect for statistically nonsignificant results”, “claiming an effect for no statistically analyzed results” and “Inappropriate extrapolating results from specific outcome to global improvement” were also relatively common.,4 out of the 5 RCTs with statistically nonsignificant results for primary outcomes had high level of spins. Moreover, the severity of spin seemed to be independent of the impact factor.

Keywords: Rosacea; Spin; RCT; Topical