I am writing to you regarding the recent publication (December 2022) by the Instituto de Evaluación Tecnológica en Salud (Institute for Health Technology Assessment - IETS by its acronym in Spanish) of the Documento técnico para el desarrollo de una Revisión Sistemática de Literatura de Efectividad y Seguridad de grupo para aplicaciones médicas de cannabis y productos terminados derivados del cannabis (Technical paper for the development of a group systematic literature review on efficacy and safety of the medical use of cannabis and cannabis-derived finished products).¹ Said technical document refers to the development of a systematic review (SR); however, according to what is reported therein, it is evident that it is a search for SRs,¹ so it should be better considered as a review of reviews.^2,3

On the other hand, the IETS technical document states that the research question to be answered by the systematic literature review (SLR) is: What is the efficacy/effectiveness and safety of the medical use of cannabis and its derivatives, compared to placebo or other drugs with current approval for the same indications? Thus, based on the research question asked and in line with the methodologies used for the elaboration of this type of studies, in theory, the most appropriate study design would be a SR, a meta-analysis, or a review of reviews.^2,3

In this sense, it has been described that a review of reviews is an appropriate type of study to provide an overview of the information available in the literature on a specific topic, since it allows comparing and contrasting the data reported in the SRs conducted on that topic. This overview is ideal for highlighting whether the available evidence on a topic is consistent or contradictory and for exploring the reasons for the findings reported in such SRs.^9-13

The review of reviews also allows for an independent evaluation of the research questions asked in the SRs analyzed in order to identify whether they report similar results and conclusions.^6-11,14-17Taking into account these characteristics, a question arises as to whether a review of reviews is the most appropriate study design to answer the research question posed in the IETS technical document¹ versus a SR and a meta-analysis since, as stated in the document, the information presented therein is qualitative in nature and, therefore, it is not clear whether it is possible to answer the question based on the evidence obtained from qualitative reviews, as is the case with a review of reviews.

Furthermore, in the IETS technical document,¹ the research question is not presented following the methodology usually used in SRs, namely the PICO model.¹⁸ In this specific case, given the large number of indications described in the document, it is likely that a single research question does not cover the specificities of each of the populations and interventions to be evaluated. For this reason, it is advisable to individualize the subgroups to be analyzed by means of several PICO questions which, in the case of qualitative studies, could have variations that may alter the results obtained.^19-26

Regarding the formulation of the objectives, the IETS technical document notes that its general objective was "to evaluate the efficacy/effectiveness and safety of the medical use of cannabis and its derivatives through a systematic literature review". This is incorrect because, as stated above, the structure of the study is actually that of a review of reviews.

Furthermore, its specific objectives show important shortcomings since, for example, the first and second specific objectives are actually the general objective divided into two, and both of them even have the same verb, which could suggest that these specific objectives are superfluous. For the third specific objective, 2 verbs are used, and although one is related to the results obtained in the SLR (to describe), the other (to evaluate) is already contemplated in the general objective and, therefore, in the first and second specific objectives.¹ This is highly relevant since the proper selection and wording of the specific objectives is very important for the choice of the study design. Thus, it is evident that some shortcomings in the design and planning of the IETS technical document can be identified even in the objectives.¹

Regarding the inclusion and exclusion criteria, the IETS technical document specifies that documents published as abstracts or published in congresses without peer review were excluded, but it also states that one of the databases consulted was Open Gray, which is widely used for the search of grey literature. In this sense, taking into account that gray literature refers to a set of documents with a wide variety of typologies that are not published or that are published but distributed through unconventional channels (conference proceedings, research reports, memoirs, projects, patents, standards, etc.),²⁷ it is evident that the inclusion of this database represents a potential selection bias.

Concerning the parameters evaluated in each study included in the SLR and considering the existing methodologies for the evaluation of the quality of the evidence of studies included in the SRs (GRADE, RoB, QUADAS, NOS, CASP, etc.), it can be noted that the IETS technical document did not contemplate items that could be useful for the analysis of the information. These comprise the objectives of the SRs included, the main characteristics of the participants, the settings and contexts evaluated, the number of studies included in each review, and the instruments used to assess the quality of the evidence in each review.

Finally, while the IETS technical document¹ made an effort to synthesize the large amount of information available on the efficacy and safety of the medical use of cannabis and its derivatives, the results are not presented in a way that allows the reader to easily identify the types of existing interventions and their outcomes. In this respect, Aromataris et al. ⁶ suggest using a stoplight indicator that reports which intervention is beneficial or effective with a green band, which intervention is inconclusive with a yellow band, and which intervention is definitely less effective relative to the comparator with a red band.