So, let's fill in the boxes one by one:
- New Outcome: "Carer Mood", this was referred to as the primary outcome that the study reported. Other outcomes were secondary outcomes, one of which was reported based on the scores in the MCTS. Carer mood was measured on how the carer scored on the MCTS scale but on a binary scale (whether for any item the score as 2 or more)
- Short Name: "Mood"
- Measured With: "Binarised MCTS score on any item to be 2 or more" (read the entire section on Methods to learn why this was so)
- Length of Follow up: "24 Months" (see above figure to learn more about it). Note that we have left the leftmost box blank as the authors in our study provided sufficient information that the entire length of follow up from randomisation was 24 months
- Type of outcome: "Dichotomous" as people were labelled on the basis of whether they reported a score of 2 or more on any one question (or one of the 10 traits they were asking them)
- Type of outcome (study): "Single study" we are abstracting data only from a single study (See Figure \ref{873161}).
- After this, click on the "save icon" to save these changes and focus on quality assessment and summary of findings boxes
We will now fill in the quality assessment and the summary of findings boxes. In order to do this, we will need to read both the methods and the results section of the paper. The first few entries are as follows:
- Number of studies: "1"
- Study Design: "Randomized Trial"
- Risk of Bias: Now, here it becomes interesting. When you review randomized controlled trials, think of how randomisation was done; in this case, they described how it was done (See Figure \ref{304528} ). This study was single blinded. Based on this information, you may label it as "not serious". If, on the other hand, you have reservations about some aspects of study design (selection or response bias), you will need to move it to serious or very serious bias.
- Inconsistency: Are the findings inconsistent? When you deal with more than one study, or say a meta analysis, check the tests of heterogeneity. The tests of heterogeneity test whether the studies were similar to each other, and also whether the results follow a simple pattern and are similar or not similar at all across the figures and numbers. In a single study such as this, if there is only one outcome and one set of metrics are reported, then this is not an issue and you would select "not serious". But if you find that a number of different outcomes were selected and the figures and numbers were varying across these outcomes, then you take a judgment call, and do a selection. In our case, for this outcome, only one set of measurement was done and therefore, there is no scope of inconsistency. So, we select, "not serious".
- Indirectness: Here, we decide whether the outcomes were indirectly measured. We will find this information from the methods section and reviewing how the outcome was measured. In this case, the outcome was carer mood and we could see it was measured using a questionnaire or survey instrument that would be self reported. This is sufficient information for us to see that the end users or people to whom the results of this study would be directed or the people from whom the data were collected, the outcomes were directly obtained from them. Hence, we select, "not serious". On the other hand, if we were to find that the outcomes were so measured that we would not get direct information from the respondents, or some other surrogate measure would be used to deduce the outcome, then, we would label it as serious or very serious depending on the specific situation. A helpful checklist and fill-in set of boxes can be accessed from "Assess directness" option and you can then fill in the set of boxes that result in the window that pops up.
- Imprecision: This is where we will test the precision estimate of the findings. We will find this information from the results section and the graphs and tables in the results section. For this article, the results section and tables are provided below (Figures \ref{437536} and \ref{873749}). You can see from the description of the results, the results did not attain a statistically significant difference. This could be because the sample size is small; but also check out the point estimate and the relevant confidence interval. If the confidence interval selected by the study authors straddle the null estimate, then the study results are imprecise. The null estimate for those outcomes that are measured on continuous scale is "0", and for those outcomes that are measured as Odds Ratios or Relative Risks is "1". So, in this case, we see that the Odds Ratio reported for the primary outcome is: OR = 0.65 (95% Confidence Interval: 0.31-1.36, p=0.25): this is for the longest term outcome (24 months). You also see that the p-value is non-significant. We advise that you do not pay too close attention to the p-value where 95% (or another) confidence interval is presented; also, if the margin is too wide, then there is a case for imprecision. After you put in a clause where you write anything but "not serious", the system will prompt you to type explanations. Write an explanation as this helps to read the tables later.
- Other Conditions: The final list in the quality assessment list is the "other conditions" that have four entries in it. We will fill in these conditions for this paper and explain:
- Publication Bias: There is no question of publication bias for a single study. If undetected, select "not detected" basically meaning "not detectable" or "not relevant". But if you deal with a meta analysis, you will need to check out if the study authors have reported publication bias or tested for publication bias. If they have, make the appropriate choice. If not, and if you suspect that the authors should have checked for publication bias, indicate that instead.
- Large Effect: A large effect is subjective. In general, consider an effect as large depending on the context where you are conducting the analysis. Something of a Relative Risk of 3.0 (or alternatively a relative risk of 0.3), or a large magnitude of risk difference (review the discussion section of the paper for validation) might be used as a guide. In our case, we see that the effects were small, and hence we select "no" here.
- Plausible Confounding: By design, if you work with Randomised controlled trials, or meta analyses, plausible confounding should be set to "no". For observational studies, check the results section and within there, check the subsection on multivariate analyses. If the point estimates drop after adjustment for potential confounders, then indicate that "would reduce demonstrated effect"; if the authors have not reported confounders, you can set it to indicate "spurious effects" but this would be relevant only for the specific outcome that you are working on. In our case, we set it to "no"
- Dose-response gradient: The dose response gradient indicates the change you observe with changing dose of the exposure or intervention. With binary variables such as the one we are studying, this is not relevant, so the default answer is "no". This is relevant for observational studies where the authors have reported doses and responses. Check the relevant graphs and decide what is the best answer here. In this case, it is "no"
Soon as you complete these boxes, you will see that GRADE will put anywhere between one and three plus signs in the quality rating for this particular outcome. A quality rating of one plus indicates that you will need more studies to justify the association obtained, and a rating of four plus would indicate that the quality of the evidence for this outcome intervention pair is sufficient at this time not to seek additional studies or that, additional studies, no matter how comprehensive will be unlikely to make a big difference in the nature of the association.
We now move to the summary of findings set of boxes. We fill in the following boxes:
- Number of patients (in our case, we start with intervention): You will find this information in the "Results" section of most papers; if this is not narratively presented, look for tables. As you expand that, you also see that the programme would want you to enter how many of them had the event (that is, the "desired event" in this case of behaviour with MCT score of 2+). In our case, we will look into Table 2 of the paper (see our Figure \ref{962877}). We will only focus on the final column as we were originally interested to find out the final tally after 24 months. So, for us, although we know that 173 people participated in the study, we are interested in the "84" people from whom we know data were obtained after 24 months. Out o them 27 people tested for positive for the MCTS behaviour score of 2+; so, when we fill in for the participants for the intervention group, we type "84" for total number of participants in the intervention arm, and "27" for those who developed the "event" (in this case MCTS score of 2+)
- You can continue to do the same thing for "controls", that is, in our case "Treatment as Usual" (TAU). But here in addition to how many developed the event out of how many controls, we also get to enter a sense of what would high, low, and/or moderate risk estimates. We do not have this information in this paper. But in some cases, you might be interested in the baseline risk of certain behaviours in the control population and would likely to be interested to check if the intervention has addressed the extent to which these behaviours can be controlled or fostered using the intervention. It is not "mandatory" or "required" for you to enter these numbers, so move on. If you have at least some values, enter it. For example, for this population, the baseline risk of bad behaviours was 40%, and you may consider it to be a high enough percentage.
- Effect: Moving on, we enter the Odds Ratio and the associated 95% Confidence Interval estimates. The programme automatically calculates the absolute effect and pro-rates this to 1000; you can change these things by clicking on the box and setting it to other bases (such as 100, or others).
- Importance: Finally, we indicate what is the importance of this particular outcome for our decision making. Let's say we consider this outcome to be critically important. What is critically important versus what is unimportant is, as you can see, subjective. A way to estimate this is to conduct a poll among the experts and seek their opinion. Often, when clients are involved, their opinions are taken into account. Say you are working for the government and you'd seek the opinion of the governnment doctors and others to seek as to what may be the combined or average score for what may be critical to important to completely irrelvant. Here, for instance, we will give it a grade of important. Could this be critical?
This exercise completes our construction of the evidence profile. Your next step is to get it to a publishable unit. Go to Step 5 (\ref{813305}).