info@biomedres.us   +1 (502) 904-2126   One Westbrook Corporate Center, Suite 300, Westchester, IL 60154, USA   Site Map
ISSN: 2574 -1241

Impact Factor : 0.548

  Submit Manuscript

Short CommunicationOpen Access

A Further Examination of Excesses or Deficits of Terminal Zeroes in Scientific Research: Using Binomial Testing to Assess Probabilities for Different Anomaly Classifications as a Function of Different Sample Sizes of Coefficients with Terminal Digits Volume 52- Issue 4

Walter R Schumm*

  • Department of Applied Human Sciences, Kansas State University, USA

Received: August 19, 2023;   Published: September 06, 2023

*Corresponding author: Walter R Schumm, Department of Applied Human Sciences, Kansas State University, 1700 Anderson Avenue, Manhattan, KS, USA

DOI: 10.26717/BJSTR.2023.52.008293

Abstract PDF

ABSTRACT

One feature recognized as a possible indication of fraudulent research is a deficit or excess of zeroes as terminal digits, compared to an expected rate of ten percent, in coefficients reported for regression and standard error coefficients. Some have translated various percentages of zeroes as terminal digits into an anomaly measure (e.g., fewer than three percent or more than twenty percent of zeroes as terminal digits might be classified as a “major” anomaly while 3.01 to 5.00% or 15.00 to 19.99% might be classified as a “moderate” anomaly). However, it has not been clear if such ordinal classifications have any meaning in terms of probability or statistical significance. The probability levels of various anomaly levels are presented and illustrated by an analysis of terminal zeroes in 16 research articles published by Dr. Eric Stewart, who has been accused of fabrication of data or results in several of his scientific publications. It is unlikely to find low probability levels (< .05) for coefficient samples of less than 50 cases but moderate or major anomalies usually have low probabilities for samples of more than 50 to 100 cases. In the case of Dr. Stewart, more than a dozen of his published articles featured probabilities below 5% with respect to zeroes as terminal digits. Using percentage classifications to create an anomaly measure for terminal digits appears to be useful when cases exceed fifty or sixty.

Introduction

There are many ways to try to detect if research has been manipulated or fabricated [1,2]. In some cases, probability formulas can assist in such efforts [3]. An excess or deficit of terminal digits in regression coefficients and their standard errors may signal data manipulation or even fabrication [4,5] because terminal digits should be approximately random (i.e., zero would occur about 10% of the time). Recently, Dr. Eric Stewart has been accused of fabricating his data or results [6-8] and was fired from his position as a full professor at Florida State University in March 2023. One of the factors involved in detecting anomalies in his publications was a frequent deficit or excess of zeroes as terminal digits in his results [9-11] where anomalies in terminal digits were classified by percentage; major anomaly (3% or fewer zeroes or 20% or more zeroes, moderate anomaly (3.01-5.00% or 15% to 19.99%), or slight anomaly (5.01 to 6.99% or 13.01 to 14.99%). The advantage of classification by percentage is that, regardless of the number of coefficients involved, the same standard can be applied to all articles in question. However, it is probable that with very small samples of coefficients, percentage-based classifications might yield anomaly classifications that were not statistically significant.

Objectives

First, binomial tests will be used to assess the validity of percentage- based anomaly classifications for different sample sizes of coefficients with terminal digits. Second, anomaly classifications assigned to articles authored by Dr. Stewart will be tested to determine how many might have been misclassified.

Methods

The most obvious issue with terminal digits that do not end in zero would be when there were no digits ending in zero or perhaps only one or two at most. But under what sample sizes (of coefficients) would such small numbers be lower than the expected ten percent and to also be statistically significant at levels of p < .05 or p < .10? Since the expected probability of a random digit is ten percent and since terminal digits in regression/standard error coefficients are approximately random, binomial tests can be used to assess the probabilities of given levels of occurrence of terminal digits. Here the focus will be on zeroes as terminal digits since such zeroes appear to be associated with fabricated research [4,5] and have been common in Dr. Stewart’s research [9-11]. Specifically, if we have 50 or any other number of coefficients, we can use binomial tests to calculate the probability of obtaining any number of zeroes or range of zeroes as terminal digits.

Using binomial tests [www.socscistatistics/tests/binomial/default2. aspx], it is apparent that 0 terminal digits of zero out of 29 or more cases/coefficients would be significant (p < .05) or 0/22 or more close to significant (p < .10). Likewise, combinations of 1/46 and 1/38 or 2/61 and 2/52 will yield similar levels of probability. At the same time, a ratio of 0/20 might seem unusual, but it would not be statistically significant. Anomaly levels created as ordinal measures to assess indications of possibly fabricated data or results will be evaluated for coefficient sample sizes of 100, 200, 300, and 500. Sixteen articles by Dr. Eric Stewart that were assigned moderate or major anomaly levels because of unusual patterns of terminal digit zeroes will be re-evaluated in terms of probability levels.

Results

What would the significance levels be for the classifications used by Schumm and his colleagues [9-11] for different sample sizes of coefficients evaluated? In (Table 1), results are provided for unusually low percentages of terminal digits ending in zero, for sample sizes of coefficients of 100, 200, 300, and 500. In (Table 2), results are provided for unusually high percentages of terminal digits ending in zero, for sample sizes of coefficients of 100, 200, 300, or 500 (Table 3). Schumm and his colleagues used a range of 7-13% to represent a normal level of terminal digit zeroes. Was that accurate? For n = 100, 200, 300, and 500 the probability of an outcome falling in that range are, respectively, .60, .80, .90, and .97. In all four examples, a majority of the time, the expected outcome was within the range of 7-13%.

Table 1: Probabilities associated with lower than expected percentages of zeroes as terminal digits.

biomedres-openaccess-journal-bjstr

Table 2: Probabilities associated with higher than expected percentages of zeroes as terminal digits.

biomedres-openaccess-journal-bjstr

Table 3: Classifications of Stewart Publications with Moderate to Major Anomalies for Terminal Digits and Associated Significance Levels.

biomedres-openaccess-journal-bjstr

Discussion

Even with a lack of zeroes as terminal digits, a sample of coefficients must be at least of 29 tests to yield a probability below .05. With one or two zeroes, samples of 46 or 61 coefficients, respectively, are needed for a probability below .05. For samples of 100, major anomalies, as defined by Schumm et al. [6-8], would have probabilities < .01, while moderate anomalies would have probabilities < .05. For samples of 300 coefficients, slight anomalies would have probabilities < .05. Approximately, levels of anomalies correspond to their statistical probabilities. As a practical illustration, articles by Stewart and his colleagues that had 40 or more coefficients and had been rated as having moderate or major anomalies with respect to zeroes as terminal digits also featured probabilities below .05, while those articles with fewer than 35 coefficients usually featured probabilities > .05 even if rated as having moderate or major anomalies with respect to zeroes as terminal digits.

In conclusion, for samples with more than 50 coefficients that can be tested for terminal digits, ratings of moderate to major anomalies appear to make statistical sense. For samples of 200 or more coefficients, even ratings for slight anomalies will feature relevant trends, with probabilities below .10, even below .05 for samples of 300 or more coefficients. These results can help future researchers more quickly evaluate apparent discrepancies in percentages of terminal digits, especially in terms of zeroes as terminal digits. While some of the anomaly classifications used by Schumm and his colleagues [7-9] for evaluating Stewart’s research did not feature probabilities below .05, it may be reasonable to consider a nearly complete lack of terminal zeroes as an issue of concern, even when not significant, in the context of numerous articles with far larger samples of coefficients that also featured few, if any, or large excesses of terminal zeroes.

If a scholar publishes large numbers of articles in which there are unusual percentages of zeroes as terminal digits (either too many or too few compared to the expected ten percent) even articles with [6- 11] so few cases as to be of only moderate probabilities may contribute to an overall pattern of anomalies in terms of zeroes as terminal digits. Probabilities associated with lower or higher than expected percentages of zeroes as terminal digits are one among other indications of anomalies that might signal scientific misconduct, even fabrication of data or results [12-27].

References

  1. Schumm Walter R, Duane W Crawford, Lorenza Lockett, Abdullah AlRashed, Asma bin Ateeq (2021) Nine ways to detect possible scientific misconduct in research with small (N < 200) samples. Psychology Research and Applications 3(2): 29-40.
  2. Schumm Walter R, Duane W Crawford, Lorenza Lockett (2019) Patterns of means and standard deviations with binary variables: A key to detecting fraudulent research. Biomedical Journal of Scientific & Technical Research 23(1): 17151-17153.
  3. Schumm Walter R (2018) Making sense of probability: What formulas to use under different conditions. Biomedical Journal of Scientific & Technical Research 3(1): 2983-2984.
  4. Mosimann James E, Claire V Wiseman, Ruth E Edelman (1995) Data fabrication: Can people generate random digits? Accountability in Research 4(1): 31-55.
  5. Mosimann James E, John E Dahlberg, Nancy M Davidian, John W Krueger (2002) Terminal digits and the examination of questioned data. Accountability in Research 9(2): 75-92.
  6. Pickett Justin T (2020) The Stewart retractions: A quantitative and qualitative analysis. Econ Watch Journal 17(1): 152-190.
  7. Breznau Nate (2021) Does sociology need open science? Societies 11(1): 9-25.
  8. Savolainen Jukka (2023) Unequal treatment under the flaw: race, crime, and retractions. Current Psychology online advance.
  9. Schumm Walter R, Duane W Crawford, Lorenza Lockett, Asma bin Ateeq, Abdullah AlRashed (2023) Can retracted social science articles be distinguished from non-retracted articles by some of the same authors, using Benford’s Law or other statistical methods? Publications 11(14): 1-13.
  10. Schumm Walter R (2023) Assessing last decimal digits in retracted research articles: Accounting for missing zeroes. SCIREA Journal of Sociology 7(4): 276-286.
  11. Schumm Walter R, Duane W Crawford, Lorenza Lockett, Abdullah AlRashed, Asma bin Ateeq (2023) Research anomalies in criminology: How serious? How extensive over time? And who was responsible? Accountability in Research, online advance.
  12. Stewart Eric A, Ronald L Simons, Rand D Conger (2000) The effects of delinquency and legal sanctions on parenting behaviors. In: Greer Litton Fox and Michael L. Benson (Eds.)., Families, crime, and criminal justice Samford, CT: JAI Press, pp. 257-279.
  13. Munoz Ed A, David A Lopez, Eric A Stewart (1998) Misdemeanor sentencing decisions: The cumulative disadvantage effect of “gringo justice”. Hispanic Journal of Behavioral Sciences 20(3): 298-319.
  14. Mears Daniel P, Eric A Stewart, Patricia Y Warren, Ronald L Simons (2017) Culture and formal social control: The effect of the code of the street on police and court decision-making. Justice Quarterly 34(2): 217-247.
  15. Stewart Eric A, Ronald L Simons, Rand D Conger, Laura V Scaramella (2002) Beyond the interactional relationship between delinquency and parenting practices: The contributions of legal sanctions. The Journal of Research in Crime and Delinquency 39(1): 36-59.
  16. Stewart Eric A, Kirk W Elifson, Claire E Sterk (2004) Integrating the general theory of crime into an explanation of violent victimization among female offenders. Justice Quarterly 21(1): 159-181.
  17. Intravia Jonathan, Kevin Wolff, Eric A Stewart, Ronald L Simons (2014) Neighborhood-level differences in police discrimination and subcultural violence: A multilevel examination of adopting the code of the street. Journal of Crime and Justice 37(1): 42-60.
  18. Mears Daniel P, Christina Mancini, Eric A Stewart (2009) Whites’ concern about crime: The effects of interracial contact. The Journal of Research in Crime and Delinquency 46(4): 524-552.
  19. Berg Mark T, Eric A Stewart, Christopher J Schreck, Ronald L Simons (2012) The victim-offender overlap in context: Examining the role of neighborhood street culture. Criminology 50(2): 359-390.
  20. Stewart Eric A, Ronald L Simons, Rand D Conger (2002) Assessing neighborhood and social psychological influences on childhood violence in an African-American sample. Criminology 40(4): 801-830.
  21. Berg Mark T, Eric A Stewart, Rod K Brunson, Ronald L Simons (2012) Neighborhood cultural heterogeneity and adolescent violence. Journal of Quantitative Criminology 28(3): 411-435.
  22. Tomlinson Tiffaney, Daniel P Mears, Jillian J Turanovic, Eric A Stewart (2021) Forcible rape and adolescent friendship networks. Journal of Interpersonal Violence 36(9-10): 4111-4136.
  23. Stewart Eric A, Ronald L Simons (2006) Structure and culture in African American adolescent violence: A test of the “code of the street” thesis. Justice Quarterly 23(1): 1-33.
  24. Mears Daniel P, Eric A Stewart, Sonya E Siennick, Ronald L Simons (2013) The code of the street and inmate violence: Investigating the salience of imported belief systems. Criminology 51(3): 695-728.
  25. Stewart Eric A, Christopher J Schreck, Ronald L Simons (2006) “I ain’t gonna let no one disrespect me”. Does the code of the street reduce or increase violent victimization among African American adolescents? The Journal of Research in Crime and Delinquency 43(4): 427-458.
  26. Berg Mark T, Eric A Stewart, Endya Stewart, Ronald L Simons (2013) A multilevel examination of neighborhood social processes and college enrollment. Social Problems 60(4): 513-534.
  27. Montes Andrea N, Daniel P Mears, Eric A Stewart (2020) Racial and ethnic divides in privatized punishment: Examining disparities in private prison placements. Justice Quarterly 37(5): 930-954.