mischbeck/MiningTransparencyManuscript
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

fixes output conditions

Browse Source
This commit is contained in:
Michael Beck
2025-12-10 16:34:49 +01:00
parent 1110f67fe3
commit aa3942532a
2 changed files with 16 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
index.qmd
+15 -17
View File
@@ -92,13 +92,11 @@ The term crisis not only implies alarmingly high proportions, but also creates p
## Questionable Research
Publication bias is the preference for publishing positive over negative or inconclusive results [@rosenthalFileDrawerProblem1979]. This bias, often called the 'file drawer problem' due to the rarity of submitted null findings, can occur at any stage in research [@kuhbergerPublicationBiasPsychology2014; @francoPublicationBiasSocial2014]. Other contributing practices include selective reporting, where null findings or variables are omitted from analysis [@breznauDoesSociologyNeed2021] and the post-hoc adaptation of hypotheses [@gerberPublicationBiasEmpirical2008]. But @breznauObservingManyResearchers2022 don't see publication bias as the main driver of the huge variance in results. Instead, they emphasize the role of idiosyncratic researcher variability and the broader context of research practices, leads to the problem of science practices that might produce unreliable or invalid results: so-called questionable research practices (QRP). In their excellent manifesto for reproducible science, @munafoManifestoReproducibleScience2017 sum up all threats to the scientific method in a single graph (@fig-qrp-circle).
!["Threats to reproducible science." @munafoManifestoReproducibleScience2017, p. 2](img/41562_2016_Article_BFs415620160021_Fig1_HTML.png){#fig-qrp-circle}
Publication bias is the preference for publishing positive over negative or inconclusive results [@rosenthalFileDrawerProblem1979]. This bias, often called the 'file drawer problem' due to the rarity of submitted null findings, can occur at any stage in research [@kuhbergerPublicationBiasPsychology2014; @francoPublicationBiasSocial2014]. Other contributing practices include selective reporting, where null findings or variables are omitted from analysis [@breznauDoesSociologyNeed2021] and the post-hoc adaptation of hypotheses [@gerberPublicationBiasEmpirical2008]. But @breznauObservingManyResearchers2022 don't see publication bias as the main driver of the huge variance in results. Instead, they emphasize the role of idiosyncratic researcher variability and the broader context of research practices, leads to the problem of science practices that might produce unreliable or invalid results: so-called questionable research practices (QRP).
A truth-incentivizing survey of over 2000 psychologists revealed a high prevalence of QRPs. Around 60% admitted to not reporting all dependent measures, 50% to selective reporting, and 30% to falsely claiming they predicted an unexpected finding. About 2% even confessed to data falsification [@johnMeasuringPrevalenceQuestionable2012a]. Criminology shows similar patterns, though with lower rates due to the absence of incentives [@chinQuestionableResearchPractices2023].
Common QRPs include HARKing or presenting an unexpected exploratory finding as a preplanned hypothesis, p-hacking or manipulating data or analysis to achieve a desired p-value and selective reporting, that is not reporting studies or variables that lack significant results. Other QRPs involve undisclosed data exclusion, stopping data collection when a desired result is found, or not reporting all conditions or measures used. These practices inflate false-positive rates and undermine research credibility [@auspurgAusmassUndRisikofaktoren2014; @breznauDoesSociologyNeed2021; @chinQuestionableResearchPractices2023].
In their excellent manifesto for reproducible science, @munafoManifestoReproducibleScience2017 conceptionalize QRPs along the typical stages of empirical research. Common QRPs include HARKing or presenting an unexpected exploratory finding as a preplanned hypothesis, p-hacking or manipulating data or analysis to achieve a desired p-value and selective reporting, that is not reporting studies or variables that lack significant results. Other QRPs involve undisclosed data exclusion, stopping data collection when a desired result is found, or not reporting all conditions or measures used. These practices inflate false-positive rates and undermine research credibility [@auspurgAusmassUndRisikofaktoren2014; @breznauDoesSociologyNeed2021; @chinQuestionableResearchPractices2023].
Other problematic practices involve the misuse of p-values, where researchers simply misinterpret the significance level as the likelihood of truth in their findings, leading to vast overconfidence in their results-that can also be a consequence of or lead to a failure to control for bias and poor quality control [@breznauDoesSociologyNeed2021; @munafoManifestoReproducibleScience2017]. Demographic, geographic or political biases and peer review limitations are more sources for error [@breznauDoesSociologyNeed2021; @grossmannOpenScienceReform2021]. Additionally, gendered penalties favor men publishing disproportionately more than women @akbaritabarGenderPatternsPublication2021. Misaligned institutional incentives, also accelerated by an intense competition for academic jobs, tenure and funding, lead to a so-called "publish or perish" culture [@smaldinoOpenScienceModified2019; @breznauDoesSociologyNeed2021].
@@ -231,7 +229,7 @@ table <- result %>% as.tibble() %>%
select(-lwr,-upr,-p) %>%
t()
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
table %>% kable()
} else {
print("Table: Estimated Minimum Sample Size")
@@ -273,7 +271,7 @@ summary_tbl <- tibble(
`Required Sample Size` = required_ns
)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
summary_tbl %>% kable(digits = 0)
} else {
print("Table: Estimated Minimum Sample Sizes - Open Science Practices")
@@ -320,7 +318,7 @@ table_sampl_est <- result %>% as.tibble() %>%
select(-lwr,-upr,-p) %>%
t()
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
table_sampl_est %>% kable(digits = 2)
} else {
print("Table: Expected 95% CI for Open Access")
@@ -375,7 +373,7 @@ tbl_cases <- tbl %>%
arrange(step_id) %>%
select(step_id, step_label, n_before, n_after, n_dropped)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_cases %>%
kable(
format = "latex", # force LaTeX output (not markdown)
@@ -543,7 +541,7 @@ tbl_cases2 <- tbl2 %>%
arrange(step_id) %>%
select(step_id, step_label, n_before, n_after, n_dropped)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_cases2 %>%
kable(
format = "latex", # force LaTeX output (not markdown)
@@ -744,7 +742,7 @@ tbl_sample_desc <- df %>% mutate(
latex.use_longtable = TRUE
)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_sample_desc
} else {
#tbl_sample_desc %>% as_kable()
@@ -960,7 +958,7 @@ overall_osp_si <- overall_osp_si %>%
) %>%
arrange(desc(`Prevalence`))
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
overall_osp_si %>%
kbl(
format = 'latex',
@@ -1093,7 +1091,7 @@ colnames(osp_table_pretty) <- c(
"Se$^{1}$", "Sp$^{2}$", "Pos$^{3}$", "Neg$^{4}$"
)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
osp_table_pretty %>%
kbl(format = "latex", booktabs = TRUE, escape = FALSE,
align = c("l","l","l","r","r","r","r"), longtable = TRUE) %>%
@@ -1367,7 +1365,7 @@ tbl_osp_prev_overall_dsadj_a <- tbl_osp_prev_overall_dsadj %>%
kable(digits = 2, row.names = FALSE, booktabs = TRUE, caption = "Preregistration") %>%
kable_styling(position = "center", full_width = FALSE)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_osp_prev_overall_dsadj_a
} else {
print("Table: Preregistration")
@@ -1400,7 +1398,7 @@ tbl_osp_prev_overall_dsadj_b <- tbl_osp_prev_overall_dsadj %>%
kable(digits = 2, row.names = FALSE, booktabs = TRUE, caption = "Open Data") %>%
kable_styling(position = "center", full_width = FALSE)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_osp_prev_overall_dsadj_b
} else {
print("Table: Open Data")
@@ -1437,7 +1435,7 @@ tbl_osp_prev_overall_dsadj_c <- tbl_osp_prev_overall_dsadj %>%
kable(digits = 2, row.names = FALSE, booktabs = TRUE, caption = "Open Materials") %>%
kable_styling(position = "center", full_width = FALSE)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_osp_prev_overall_dsadj_c
} else {
print("Table: Open Materials")
@@ -1469,7 +1467,7 @@ tbl_osp_prev_overall_dsadj_d <- tbl_osp_prev_overall_dsadj %>%
kable(digits = 2, row.names = FALSE, booktabs = TRUE, caption = "Open Access") %>%
kable_styling(position = "center", full_width = FALSE)
if(output_format == "pdf") {
if(output_format == "pdf/tex") {
tbl_osp_prev_overall_dsadj_d
} else {
print("Table: Open Access")
@@ -1497,7 +1495,7 @@ if (isTRUE(debug_mode)) {
#| results: asis
if (isTRUE(debug_mode)) {
print("# Debug Info\n\n")
print("# Debug Info")
print(debug_info)
}
```