Clinical Trials and Network Meta-Analysis (NMA)

Written by on September 30, 2013 in Features - No comments

Side by Side


Drug development is a term used to define the process of bringing a new drug to the market when a new molecular entity has been identified through the process of drug discovery.(1,2) Drug development phase is followed by the regulatory approval phase (marketing authorization). Once the drug is on the market, market access is the fourth hurdle that it needs to overcome. When the drug is approved and reimbursed the standard lifecycle phases like marketing, sales and pharmacovigilance (monitoring of adverse drug reactions) are implemented.

In contrast to what is commonly believed, drug development is not limited to preclinical development (in laboratories, animal experiments) and evaluation in clinical trials (human subjects). Drug development may include the process of obtaining regulatory approval to market the new compound. In order to bring a single new drug to the market, companies are screening and synthetizing between 5 and 10 thousand candidate compounds in the drug discovery process. Most of these compounds get discarded at early stages, but some get further up the ladder. In drug development, all parameters of the future drug (safety, pharmacokinetics, pharmacodynamics, mechanism of action) are assessed prior to exposure to humans in clinical trials. The most important features of a new drug are that it is effective (i.e. treats a disease) and that it is safe. Out of 10,000 newly synthetized molecules about 250 on average show promise for pre-clinical research, of which only 10 qualify for tests on humans.(3) This huge number of compounds initially screened translates into high costs to only identify a possible drug candidate.


Clinical trials are sets of tests in medical research that provide data about quality, safety and efficacy of a new drug. They are intended to explore or verify the effects of one or more investigational drugs.(4) Clinical trials are actually designed for regulatory purposes, to evaluate quality, safety and efficacy of a new drug. For the purpose of market access and reimbursement a different set of questions need to be answered like: does the drug have added clinical value to the ones already on the reimbursement lists, what is the budget impact of the new drug, etc?

However, the whole drug-lifecycle, from its discovery through marketing authorization till its withdrawn from the market is not just complex and long, but extremely expensive too. Study estimated the costs of bringing a new drug to the market at approximately US$800 million.(5) Estimates in other studies ranged from US$500 million to US$5 billion depending on the disease, therapy and/or the developing company.(6,7,8)

Pharmaceutical companies need to enrich their pipelines with new branded compounds, but are faced with hurdles to bring those compounds on the market, including rising costs for clinical trials and more evidence. According to a recent survey, rising costs occur across all phases of clinical trials, because of increasing competition for trial sites and clinical research organizations that can yield reliable, high quality data.(6,9) For the purpose of marketing authorization, new innovative drugs are often compared with placebo or standard therapy, but not against all available treatment options.

In an era of evidence based medicine, where more and more data is required before a drug may enter the market and more clinical trials are needed to generate data, it may be wise to explore other possibilities to obtain data for reimbursement submissions.

What could pharmaceutical companies actually do to collect as much evidence as possible to ensure the market access of the new drug?

A good solution could be a network meta-analysis (NMA). But what is NMA exactly?

Network meta-analysis, in the context of a systematic (literature) review, is a meta-analysis in which multiple treatments (three or more) are being compared using both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator (active treatments or placebo).(10) NMA therefore compares results from two or more studies that have one treatment in common. The benefit of NMA lies in the fact that the relative efficacy and/or safety of a particular drug versus its competitors can be obtained in the absence of head-to-head evidence. Using NMA enables simultaneous comparisons of virtually all drugs available on the market to treat a disease.

A simple example of a NMA would be the following: an initial trial compares drug A to drug B. A different trial studying the same patient population compares drug B to drug C (see Figure).


Figure: Illustration of a simple network used for network meta-analysis

We could assume that drug A is found to be superior to drug B in the first trial and drug B is found to be equivalent to drug C in a second trial. Since drug A is better than drug B, and drug B is equivalent to drug C, drug A is also better than drug C even though there is no clinical trial directly comparing these two drugs. On the basis of statistical inference, a NMA of these two trials would allow a researcher to conclude that treatment option A is more effective than option C, even though the two options have never been directly compared.(11)

Randomized controlled trials (RCTs) are considered as the gold standard of evaluating the efficacy and safety of a new drug compared to other drugs. However, when the available clinical trials do not compare the same drugs to each other, it is possible to develop a network of RCTs where all trials have at least one drug in common with another one. Such a network allows for indirect comparisons of drugs not studied in a head-to-head fashion.(12) In that aspect, NMA is different from classical pairwise meta-analysis in the sense that there is not only one type of treatment comparison (drug A versus B), but multiple treatment comparisons (drug A versus B, B versus C and A versus C). Since NMA allows for using a larger evidence base in the analysis, the drug can be evaluated in a broader spectrum than is possible with a single pairwise meta-analysis, which often offers a fragmented picture of the drug’s performance.(13)

A NMA combines multiple studies and makes statistical comparisons in a similar manner as done within a RCT. NMA results are more likely to be valid when analyzing studies with a similar study design and similar patient populations. Indirect estimates obtained with a NMA might be preferable to the estimates obtained by pairwise meta-analyses since the analysis is based on a broader evidence base.(14,15) However, the broader evidence base is also counterbalanced with a risk of introducing bias.

The purpose of NMA is not to reduce costs of RCTs or to diminish the value of clinical trials, but to make a larger evidence base for everybody (clinicians, pharmacists, regulatory agencies like FDA) to make informative decisions.

NMA simply informs decision problem, and it might provide insight why some treatments are better than the others. These insights might trigger further research and development programs and even better care for patients.  One example of the insights are treatment gaps that include complex relationships among culture, environment, population genetics, drug metabolism, and drug response which could be missed/overlooked in one single clinical trial, but in a NMA these issues would become apparent. Increased awareness of coexisting conditions and a better use of knowledge regarding patient response factors that impact the effectiveness and safety of drug therapy are just some examples of treatment gaps.


Clinical trials are certainly the gold standard to assess the efficacy and safety of a treatment compared to other treatments.(13) NMA allows for simultaneous indirect comparison of all available treatments, incorporating more data in the analysis ensuring optimal use of all available evidence. Comparisons of all available treatment options would be very useful to pharmaceutical companies as much as to clinicians to make informative decisions about the treatment of patients. The final costs for drug development and clinical drug evaluations are extremely high and cost reductions would be beneficial to all stakeholders. However, expanding the evidence base is crucial. In the absence of RCTs of all available treatments, or combined with RCTs, NMA would provide useful evidence for selecting the best choice of treatment and should therefore be used more often to inform decision making. NMA cannot replace clinical trials, but can complement them and can help in generalizability of the findings from the RCTs (efficacy to effectiveness).

  1. Clinical Trials Handbook“. Shayne Cox Gad (2009). John Wiley and Sons. p.118. ISBN 0-471-21388-8
  2. Curtis L. Meinert, Susan Tonascia (1986). Clinical trials: design, conduct, and analysis. Oxford University Press, USA. p. 3. ISBN 978-0-19-503568-1.
  3. Stratmann, Dr. H.G. (September 2010). “Bad Medicine: When Medical Research Goes Wrong”. Analog Science Fiction and FactCXXX (9): 20.
  4. Clinical Trials in Humans -
  5. DiMasi, Joseph A.; Hansen, Ronald W.; Grabowski, Henry G. (March 2003). “The price of innovation: new estimates of drug development costs”. Journal of Health Economics 22 (2): 151–185.
  6. Adams C, Brantner V (2006). “Estimating the cost of new drug development: is it really 802 million dollars?”. Health Aff (Millwood) 25 (2): 420–8.
  7. Adams, Christopher Paul; Brantner, Van Vu (February 2010). “Spending on new drug development”. Health Economics 19 (2): 130–141.
  8. Pharma & Healthcare. 8/11/2013. The Cost of Creating A New Drug Now $5 Billion, Pushing Big Pharma To Change
  9. Silverman E (2011), “Clinical Trial Costs Are Rising Rapidly” Pharmalot
    10. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed.BMC Med. 2011 Jun 27; 9:79.
    12. Caldwell DM, Ades AE, Higgins JPT: Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005, 331:897-900
    13. Jansen JP, Naci H.Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 2013 Jul 4; 11:159.
    14. Lu G, Ades AE: Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 2006, 101:447-459.
    15. Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR: Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods 2012, 3:98-110.

By Goran Medic, Feike van der Scheer, Eline Huisman, Gert Bergman
Mapi – HEOR & Strategic Market Access, The Netherlands
Corresponding author: Goran Medic;


Leave a Comment

Please type the characters of this captcha image in the input box

Please type the characters of this captcha image in the input box