Letter to the editor

The technical paper on efficacy and safety of the medical use of cannabis and cannabis-derived finished products is not a systematic review

El documento técnico de efectividad y seguridad para aplicaciones médicas del cannabis y productos terminados derivados del cannabis no es una revisión sistemática

Gustavo Ortega-Mora1

1 Universidad del Rosario - School of Medicine and Health Sciences - Bogotá D.C. - Colombia.

Open access

Received: 31/01/2023

Accepted: 25/03/2023

Corresponding author: Gustavo Ortega-Mora. Escuela de Medicina y Ciencias de la Salud, Universidad del Rosario, Bogotá D.C. Colombia. Email: drortegagustavo@gmail.com.

Keywords: Systematic Reviews as Topic; Review Literature as Topic; Meta-Analysis as Topic; Bias, Epidemiologic (MeSH).

Palabras clave: Revisiones sistemáticas como asunto; Literatura de revisión como Asunto; Metaanálisis como asunto; Sesgo (Epidemiología) (DeCS).

How to cite: Ortega-Mora G. The technical paper on efficacy and safety of the medical use of cannabis and cannabis-derived finished products is not a systematic review. Rev. Fac. Med. 2023;71(3):e107048. English. doi: https://doi.org/10.15446/revfacmed.v71n3.107048.

Cómo citar: Ortega-Mora G. [El documento técnico de efectividad y seguridad para aplicaciones médicas del cannabis y productos terminados derivados del cannabis no es una revisión sistemática]. Rev. Fac. Med. 2023;71(3):e107048. English. doi: https://doi.org/10.15446/revfacmed.v71n3.107048.

Copyright: Copyright: ©2023 Universidad Nacional de Colombia. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, as long as the original author and source are credited.

Dear Editor

I am writing to you regarding the recent publication (December 2022) by the Instituto de Evaluación Tecnológica en Salud (Institute for Health Technology Assessment - IETS by its acronym in Spanish) of the Documento técnico para el desarrollo de una Revisión Sistemática de Literatura de Efectividad y Seguridad de grupo para aplicaciones médicas de cannabis y productos terminados derivados del cannabis (Technical paper for the development of a group systematic literature review on efficacy and safety of the medical use of cannabis and cannabis-derived finished products).1 Said technical document refers to the development of a systematic review (SR); however, according to what is reported therein, it is evident that it is a search for SRs,1 so it should be better considered as a review of reviews.2,3

On the other hand, the IETS technical document states that the research question to be answered by the systematic literature review (SLR) is: What is the efficacy/effectiveness and safety of the medical use of cannabis and its derivatives, compared to placebo or other drugs with current approval for the same indications? Thus, based on the research question asked and in line with the methodologies used for the elaboration of this type of studies, in theory, the most appropriate study design would be a SR, a meta-analysis, or a review of reviews.2,3

In this sense, it has been described that a review of reviews is an appropriate type of study to provide an overview of the information available in the literature on a specific topic, since it allows comparing and contrasting the data reported in the SRs conducted on that topic.4-9 This overview is ideal for highlighting whether the available evidence on a topic is consistent or contradictory and for exploring the reasons for the findings reported in such SRs.9-13

The review of reviews also allows for an independent evaluation of the research questions asked in the SRs analyzed in order to identify whether they report similar results and conclusions.6-11,14-17 Taking into account these characteristics, a question arises as to whether a review of reviews is the most appropriate study design to answer the research question posed in the IETS technical document1 versus a SR and a meta-analysis since, as stated in the document, the information presented therein is qualitative in nature and, therefore, it is not clear whether it is possible to answer the question based on the evidence obtained from qualitative reviews, as is the case with a review of reviews.

Furthermore, in the IETS technical document,1 the research question is not presented following the methodology usually used in SRs, namely the PICO model.18 In this specific case, given the large number of indications described in the document, it is likely that a single research question does not cover the specificities of each of the populations and interventions to be evaluated. For this reason, it is advisable to individualize the subgroups to be analyzed by means of several PICO questions which, in the case of qualitative studies, could have variations that may alter the results obtained.19-26

Regarding the formulation of the objectives, the IETS technical document notes that its general objective was “to evaluate the efficacy/effectiveness and safety of the medical use of cannabis and its derivatives through a systematic literature review”. This is incorrect because, as stated above, the structure of the study is actually that of a review of reviews.

Furthermore, its specific objectives show important shortcomings since, for example, the first and second specific objectives are actually the general objective divided into two, and both of them even have the same verb, which could suggest that these specific objectives are superfluous. For the third specific objective, 2 verbs are used, and although one is related to the results obtained in the SLR (to describe), the other (to evaluate) is already contemplated in the general objective and, therefore, in the first and second specific objectives.1 This is highly relevant since the proper selection and wording of the specific objectives is very important for the choice of the study design. Thus, it is evident that some shortcomings in the design and planning of the IETS technical document can be identified even in the objectives.1

Regarding the inclusion and exclusion criteria, the IETS technical document specifies that documents published as abstracts or published in congresses without peer review were excluded, but it also states that one of the databases consulted was Open Gray, which is widely used for the search of grey literature. In this sense, taking into account that gray literature refers to a set of documents with a wide variety of typologies that are not published or that are published but distributed through unconventional channels (conference proceedings, research reports, memoirs, projects, patents, standards, etc.),27 it is evident that the inclusion of this database represents a potential selection bias.

Concerning the parameters evaluated in each study included in the SLR and considering the existing methodologies for the evaluation of the quality of the evidence of studies included in the SRs (GRADE, RoB, QUADAS, NOS, CASP, etc.), it can be noted that the IETS technical document did not contemplate items that could be useful for the analysis of the information. These comprise the objectives of the SRs included, the main characteristics of the participants, the settings and contexts evaluated, the number of studies included in each review, and the instruments used to assess the quality of the evidence in each review.

Finally, while the IETS technical document1 made an effort to synthesize the large amount of information available on the efficacy and safety of the medical use of cannabis and its derivatives, the results are not presented in a way that allows the reader to easily identify the types of existing interventions and their outcomes. In this respect, Aromataris et al.6 suggest using a stoplight indicator that reports which intervention is beneficial or effective with a green band, which intervention is inconclusive with a yellow band, and which intervention is definitely less effective relative to the comparator with a red band.

References

1.Colombia. Instituto de Evaluación Tecnológica en Salud (IETS). Documento técnico para el desarrollo de una Revisión Sistemática de Literatura de Efectividad y Seguridad de grupo para aplicaciones médicas de cannabis y productos terminados derivados del cannabis. Bogotá D.C.: IETS; 2022 [cited 2023 Nov 17]. Available from: https://bit.ly/3MOdL2Y.

2.Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/gfv7cz.

3.Papatheodorou SI, Evangelou E. Umbrella Reviews: What They Are and Why We Need Them. Methods Mol Biol. 2022;2345:135-46. https://doi.org/gq6t46.

4.Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91-108. https://doi.org/ftbpbr.

5.Cooper H, Koenka AC. The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. Am Psychol. 2012;67(6):446-62. https://doi.org/f4bmsr.

6.Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132-40. https://doi.org/gfkpmq.

7.Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15. https://doi.org/db7vmf.

8.Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev. 2016;5(1):190. https://doi.org/ghq7t4.

9.Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ. 2009;181(8):488-93. https://doi.org/d7rpht.

10.Pieper D, Buechter R, Li L, Prediger B, Eikermann M. Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties. J Clin Epidemiol. 2015;68(5):574-83. https://doi.org/gp57kz.

11.Biondi-Zoccai G, Lotrionte M, Landoni G, Modena MG. The rough guide to systematic reviews and meta-analyses. HSR Proc Intensive Care Cardiovasc Anesth. 2011;3(3):161-73.

12.Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667. https://doi.org/bmtg.

13.Catalá-López F, Hutton B, Núñez-Beltrán A, Page MJ, Ridao M, Macías-Saint-Gerons D, et al. The pharmacological and non-pharmacological treatment of attention deficit hyperactivity disorder in children and adolescents: A systematic review with network meta-analyses of randomised trials. PLoS One. 2017;12(7):e0180355. https://doi.org/gbm48q.

14.Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/gfv7cz.

15.Ballard M, Montgomery P. Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist. Res Synth Methods. 2017;8(1):92-108. https://doi.org/f9ktfc.

16. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/gfxrks.

17.Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/gg898p.

18.Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16. https://doi.org/dhzzhh.

19.Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123(3):A12-3.

20.Da Costa Santos CM, de Mattos Pimenta CA, Nobre MR. The PICO strategy for the research question construction and evidence search. Rev Lat Am Enfermagem. 2007;15(3):508-11. https://doi.org/b8m5gd.

21.Petticrew M, Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide. Malden, MA: Blackwell Publishing; 2006. https://doi.org/fwfpfx.

22.Noyes J, Popay J, Pearson A, Hannes K, Booth A. Chapter 20: Qualitative research and Cochrane reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011 [cited 2023 Nov 17]. Available from: https://bit.ly/3QIxylI.

23.O’Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev. 2018;7(1):3. https://doi.org/ghf9x4.

24.Gopalakrishnan S, Ganeshkumar P. Systematic reviews and meta-analysis: understanding the best evidence in primary healthcare. J Family Med Prim Care. 2013;2(1):9-14. https://doi.org/gd5493.

25.Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. https://doi.org/c73jf5.

26.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. https://doi.org/dh2cj4.

27.Farace D, Schöpfel J, esditors. Grey Literature in Library and Information Studies. Berlin: De Gruyter Saur; 2010.