VAS internal validity adjustment 2022 (outdated as of August 2023)

	A	B	C	D
1	Setting the VAS internal validity (IV) adjustment
2
3
4	Internal validity benchmark	Adjustment	Weight	Notes
5	Starting internal validity adjustment	0.85
6	Penalty from component 1: uncertainty about mechanisms	7.5%	0.5	A rough assumption about how the current IV adjustment breaks down between trial methodology concerns and mechanisms concerns.
7	Penalty from component 2: trial quality/methodology	7.5%	0.5
8
9	Component 1: internal validity adjustment for estimate plausibility
10	RCT effect size from meta-analysis	100%	0.5	Significant weight assigned because the VAS mortality benefit is ostensibly a high-quality estimate that comes from a meta-analysis of 18 RCTs, and because of high uncertainty in our mechanisms analysis.
11	Comparison with other interventions	61%	0	Reflects our lower confidence in the VAS estimates relative to bednets and SMC. No weight assigned due to lack of confidence in how to attribute VAS effects across children with/without Vitamin A deficiency, and because the mechanisms approach seems more compelling and covers some of the same concerns.
12	Mechanisms approach: measles + diarrhea mortality	64%	0.5	Benchmarks the plausibility of estimates to adding up the mortality reductions expected from intermediate causes of death. We split the weight on this mechanisms approach with the meta-analysis effect size. Assigned substantial weight because this analysis shows that we can't explain the VAS effect size using a mechanisms-based approach, which is a negative update. Retained substantial weight on a more optimistic scenario because there are weaknesses to the mechanisms approach that lead this estimate to have wide confidence intervals: uncertainty about IHME cause-of-mortality inputs; somewhat imprecise estimates for mortality/morbidity averted by measles + diarrhea; and uncertainty about our key assumptions of indirect deaths and the proportion of the VAS effect attributable to measles + diarrhea. The indirect deaths assumption is the most uncertain, because we've seen other examples of interventions where there may be surprisingly many indirect deaths averted per direct death, such as water quality interventions.
13	IV adjustment based on mechanisms and estimate plausibility	82%
14
15	Component 2: IV adjustment for trial quality concerns
16	IV penalty based on concerns about the quality of execution for VAS trials run during the 1980s-early 1990s	7.5%		Assumed to be the same as our starting value for this component. Most VAS RCTs were conducted in the 1980s and 1990s (see study dates in "Original Imdad 2017 - VAD prevalence estimates" tab), and this may affect their quality of execution. I assume that this value is additive with all other concerns about plausibility of the estimates. While these issues may not be fully independent since poor trial quality could lead to implausible estimates, I think this remains a distinct concern that warrants a small IV penalty.
17
18	Final adjustment
19	Sum of adjustments for plausibility and trial quality	74%
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100