What type of research is a meta-analysis, and what are its stages of development?
Let’s imagine that we intend to generate a new type of therapy to treat a disorder, for example borderline personality disorder. To do this, the first step to follow is to collect and assess the effectiveness of the different treatments and pre-existing methodologies for said disorder: to look for the effectiveness of the treatments in previous studies. But a simple primary study would not be valid for us, since it may be biased and may even reflect results that contradict those of others.
Thus, we need to resort to studies that have analyzed the behavior of what we want to observe throughout different studies, and for this the most reliable methodology or document is the use of meta-analysis.
What is a meta-analysis? Let’s see it throughout this article.
What is a meta-analysis?
We can define a meta-analysis as the result of carrying out a systematic and exhaustive investigation on a very specific topic or question to be answered, ** combining and assessing together the results obtained by a large number of studies on that aspect or topic **. It is an in-depth review that allows us to see with great clarity the way in which the studies it reviews have been selected and evaluated, in addition to analyzing their results and evaluating quantitatively through various statistical procedures, among which the evaluation of the effect size and control for possible publication bias.
It has the great advantage that it allows us to work with large amounts of information from a large number of studies, in a highly organized way and that allows us to work with the scientific method with large samples in a relatively simple way, in addition to assessing differential aspects between studies and sources of heterogeneity that baseline studies may not have taken into account.
However, meta-analysis is not the only existing methodology in the systematization and synthesis of research results on a specific topic : there is also the concept of systematic review, which, like meta-analyzes, reviews in an objective, systematic and reflecting its methodology in order to be able to replicate the existing data in the scientific literature existing up to now.
It is necessary to bear in mind that, in fact, technically every meta-analysis is a systematic review that incorporates the statistical analysis of the results. And this is fundamental and allows us a much greater precision in our research, for example being able to assess from statistical analysis which treatments for a certain disorder or even symptom are more consistently effective in the various studies evaluated.
When should they be used?
The use or performance of meta-analyzes can be of great help in a large number of areas, but they are much more relevant when we are faced with the need to investigate aspects in which previous studies show a certain degree of controversy and contradiction among themselves, or either when we want to investigate what degree of relationship different variables have or try to assess what size of the effect exists regarding the effectiveness of a program or treatment. Thus, in areas such as psychology or medicine, this type of review is generally used to explore, investigate, prioritize and apply treatments based on evidence.
Now, it must be borne in mind that its application is only viable when the primary studies chosen have a quantitative perspective and with a sample and conditions that are homogeneous or very similar to each other, and obviously that they start from at least the same objective and main question . It would also be irrelevant if what we are looking for are cause-consequence relationships.
Main phases in conducting meta-analysis
Performing a meta-analysis is not easy and requires the systematic monitoring of a series of steps or phases, which are detailed below.
1. Formulation of the problem / research question
The first step, probably the most important although on the other hand obvious, is to formulate or pose the problem, theme or aspect that we want to investigate and analyze. It is not a question of asking a question at random, but of defining it as clearly and concisely as possible and taking into account the various variables that we are going to have to assess.
Based on this first question, the objectives of the research will be constructed and the meta-analysis will begin to structure, and even to generate inclusion and exclusion criteria for studies.
2. Bibliographic search and selection of studies
The second of the steps, probably one of the longest, goes through the active search of the bibliography that we are going to analyze together in order to make a good review of our research topic.
To do this, it is not enough to simply take every study that we see: it is necessary first of all to have generated inclusion and exclusion criteria that allow us to select those that meet the requirements to be evaluated in the meta-analysis among those that exist.
The criteria should take into account the type of design that each study has, how the results have been measured and the main characteristics of the samples used or of the experimental methodologies. Thus, if we consider, for example, what effects a drug has on the menstrual cycle, we will not take studies in which the sample is male. Or if age is a factor to take into account in our research, we will limit ourselves to studies whose sample includes only subjects with the age that is relevant to us, or else that has at least separated the age groups.
In addition to this, other factors must be taken into account when making the selection: the language or language in which they are written (if this is relevant) or the date of publication (it is recommended not to use data from studies of more than ten years ), among others.
For the search, it is usually advisable to use indexed databases such as SCOPUS or PubMed, at the computer level, although other types of document or systems can also be used.
3. Coding of studies
Once the studies that are going to be part of the meta-analysis have been selected, it is necessary to produce a coding manual in which the reliability of the data and the variables that are taken into account in the model used are recorded: the variables must be included among them of the sample, extrinsic, methodological and contextual. In general, any moderating variable that may influence the results should be included, as well as indicate how they have been assessed so that other researchers can reach the same results using the same methods.
4. Statistical analysis and interpretation
The fourth step, in which we are already facing a meta-analysis, is the statistical analysis of the results. In this phase , the implications or interpretations of the results are evaluated and subsequently discussed.
As a general rule, this statistical analysis tends to assess the effect sizes of the different treatments or compared studies, assess whether or not there is heterogeneity (if necessary, using strategies to reduce the effect this may have on the interpretation of the data) and, if necessary, that this exists to assess which variables can explain such heterogeneity from tests such as ANOVA.
Once the interpretation of the results is finished, the last of the steps to carry out the meta-analysis is to write it and publish it, having at least the introductory sections (which include objectives and hypotheses as well as a theoretical framework), method (which must be very clear and detailed so that other authors could replicate it and should include how and where we have searched for the studies, the coding manual and the type of analysis carried out), results and discussion (where it is interpreted based on the data of the results ).
The problem of publication bias
A possible problem to take into account when assessing the results of research and even meta-analysis is that there may be publication bias, understood as the deviation in the results that can be caused by the tendency of the scientific literature to search and favor studies that show statistically significant relationships between the variables used and ignore those that do not.
That is, studies in which relationships between variables are seen are published, but those in which the results do not show a significant relationship tend to be less published or ignored. Fortunately, this effect can be visualized and evaluated in meta-analyzes, despite the fact that it is a difficult problem to solve.
- Marín Martínez, F., Sánchez Meca, J. and López López, JA (2009). Meta-analysis in the field of Health Sciences: an essential methodology for the efficient accumulation of knowledge. Physiotherapy, 31 (3): 107-114.
- Sánchez-Meca, J. and Ato-García, M. (1989). Meta-analysis: a methodological alternative to traditional research reviews. In: Arnau J, Carpintero H, editors. Treatise on General Psychology I: History, Theory and Method. Madrid: Alhambra. 7 – 69.