This project was funded by the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme (project number 05/52/01) and was published in full in Health Technology Assessment 2009; Vol.13: No.59
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA programme, NIHR, NHS or the Department of Health.
Below is a list of relevance, reliability and transferability questions to ask when considering the adaptation of information and/or data on effectiveness and efficacy (box 8).
Question Box 8: Effectiveness questions
a) To assess relevance
1. a) What is the research question considered?
b) Is the research question considered within this section of the HTA report relevant to your HTA question?
2. Are the outcome measures relevant for your HTA question?
3. Were the search methods used to find studies relevant to the main question(s) stated?
b) To assess reliability6
4. Was the search for studies reasonably comprehensive?
5. Were the criteria used for deciding which studies to include in the HTA report reported?
6. Was bias in the selection of studies avoided?
7. Did the selection of studies (in particular the choice of eligible study designs) minimise the possibility of including studies with a high propensity for bias?
8. Were the criteria used for assessing the validity of the included studies reported?
9. Was the validity of all studies referred to in the text assessed using appropriate criteria (either in selecting studies for inclusion or in analysing the studies that are cited)?
10. Were the methods used to combine the findings of the relevant studies (to reach a conclusion) reported?
11. Were the findings of the relevant studies combined appropriately with respect to the main question the HTA report addresses?
12. Were the conclusions made by the authors supported by the data and/or analysis reported in the HTA report?
13. How likely is it that the relevance of this HTA report has changed due to additional research that had started, completed or been published since this Health Technology Assessment report?
c) To assess transferability
14. Would you expect the baseline risk of patients within your own setting to be the same as the baseline risk of those patients considered within the HTA report for adaptation? (assuming that patients receive the same treatment and same comparator)
We would expect the relative risk to be the same and baseline risk different. The user needs to consider the impact of local epidemiological and demographic data on the baseline risk.
Answers to these questions should help the user extract information and/or data from this section of the HTA report. This „adaptation material‟ on effectiveness (including efficacy) can be incorporated within an HTA report in your own setting. There may be a need to update these data and supplement it with local context data.
6 The majority of these reliability questions have been taken from the „Overview Quality Assessment Questionnaire‟: Shea BJ, Boers M, Grimshaw JM, Hamel CD, Bouter LM. Does updating improve the methodological and reporting quality of systematic reviews? BMC Medical Research Methodology 2006; 6:27.