Appendix
Derivation of a relation between stimulus (effect) size and response [probability of not requiring further RCTs when\(\mathbf{jnd=}\log\mathbf{(OR)}\mathbf{]}\) based on analogy to Weber-Fechner law
Let s be the magnitude of a measurable stimulus and Δs the increase in stimulus just required to discriminate between stimuli as:21
\begin{equation} r=\frac{\Delta s}{s}=constant\nonumber \\ \end{equation}
This means that the noticeable difference in sensation occur only when the increase, or change in stimuli (such as change in the magnitude of effect due to the experimental treatment compared with control treatment) are a constant percentage of the stimulus (s ) itself. This is Weber’s law.
Fechner proposed a method of scaling that takes Weber’s law into account; let \(s_{o}\) be a fixed value of s to allow us to calculate the nearest noticeably higher stimulus as21:
\begin{equation} r=\frac{s_{1}-s_{o}}{s_{o}}\ \text{or\ }r\cdot s_{o}=s_{1}-s_{o}\ \nonumber \\ \end{equation}\begin{equation} s_{1}=s_{o}+r\cdot s_{o}=s_{o}\cdot\left(1+r\right)=s_{o}\cdot q\text{\ where\ }q=1+r\nonumber \\ \end{equation}
So the “next” stimulus is q times the previous level of “noticeable” stimulus. Similarly
\begin{equation} s_{2}=s_{1}\cdot q=s_{o}\cdot q\cdot q=s_{o}\cdot q^{2}\text{\ \ \ and\ \ }\ s_{3}=s_{2}\cdot q=s_{1}\cdot q^{2}=s_{o}\cdot q^{3}\nonumber \\ \end{equation}
which leads to general equation
\begin{equation} s_{n}=s_{o}\cdot q^{n}\nonumber \\ \end{equation}
Which, after taking a natural logarithm (or logarithm base 10) is analogous to
\begin{equation} n=A\cdot\ln s+B\nonumber \\ \end{equation}
where \(A=\frac{1}{\ln q}\) and \(B=-\frac{\ln s_{o}}{\ln q}\).
In our case, we actually only want to distinguish whether the difference in stimulus is noticeable, i.e., we are not interested in the absolute size of the stimulus, but rather whether the ratio \(r\)is large enough.
We express the ratio as effect size measured in terms of odds ratio (OR), or proportional reduction of OR (1-OR, in terms of reducing bad events); or, increase in terms of improving good outcomes (OR-1). These effects are commonly assumed to remain constant over the range of predicted risks33, providing further justification for application of Weber-Fechner law:
\begin{equation} r=\frac{\Delta s}{s}=\frac{ODDSexp-ODDSctrl}{\text{ODDctrl}}=\frac{\text{ODDSexp}}{\text{ODDSctrl}}-1=OR-1\nonumber \\ \end{equation}
and
\begin{equation} q=\left(1+r\right)=\left(1+OR-1\right)=OR\nonumber \\ \end{equation}
where ODDSexp and ODDSctrl, are odds of events related to the experimental and control treatment, respectively.
Rather than using the size of the stimulus (n), we are using the size of the ratio (\(r\) or \(q\)) as the independent variable. Since this is a ratio, the use of logarithmic scale is more appropriate, and rather than modeling the scale or intensity of the stimulus (n) as in the Weber’s law, we are modeling whether the size of this ratio will indicate a distinguishable difference :
\begin{equation} \text{diff\ }\left(\text{yes\ or\ no}\right)=A\cdot\ln\left(q\right)+B=A\cdot\ln\left(\text{OR}\right)+B\nonumber \\ \end{equation}
Since in this case, this difference can only take two values (0=no=further RCTs are required or 1=yes=further RCTs are not required), we employ logistic regression48 to obtain probability of not requiring further RCT:
\begin{equation} \text{logit}\left(P\left(\text{no}n_{\text{RCT}}\right)\right)=A\cdot\ln\left(\text{OR}\right)+B\nonumber \\ \end{equation}
Similar expression can be derived using relative risks instead of OR.