Response to the cost effectiveness analysis of pneumococcal conjugated vaccines in Peru

Dear Editor, We have read with interest the paper by MezonesHolguin et al., 2014 [1] recently published in this journal, reporting the cost effectiveness estimates for pneumococcal conjugated vaccines (PCVs) in Peru. After careful examination of the analysis, we identify some methodological inconsistencies, which we believe would make this study insufficient for decision making. A detailed discussion of these concerns is presented below.


Dear Editor,
We have read with interest the paper by Mezones-Holguin et al., 2014 [1] recently published in this journal, reporting the cost effectiveness estimates for pneumococcal conjugated vaccines (PCVs) in Peru.After careful examination of the analysis, we identify some methodological inconsistencies, which we believe would make this study insufficient for decision making.A detailed discussion of these concerns is presented below.

Outcomes considered
A number of guidelines have been published by international organizations such as the World Health Organization, Gates Foundation and National Institute for Health and Care Excellence to help countries run informed Health Technology Assessments [2][3][4].They state that economic evaluations should include all positive and negative effects of an intervention to produce balanced evidence for decision making.Health economic evaluations not including all conditions related to the intervention or with a limited time horizon might be missing positive/negative effects and hence should not be considered.
In this study, the authors only consider the effect of PCVs over pneumococcal pneumonia disregarding their proven effect over other important outcomes.As efficacy against invasive disease and acute otitis media (AOM) has been reported in several clinical trials, efforts should be made to approximate the Peruvian burden based on neighboring country values (e.g.authors are using Uruguay for serotype distribution).
We believe an economic evaluation that does not include these outcomes strongly related to infant mortality and health in Peru, biases the cost effectiveness results and limits its use to inform any decision regarding vaccine recommendation.For this reason, peer reviewed economic evaluations of PCVs worldwide and in Latin American countries (including Peru), have all included pneumonia, invasive disease and AOM [5][6][7][8] as the outcomes considered.

Vaccine efficacy estimations
The relative risk reduction or direct effect (DE, as described in the study) for PCV7 against hospitalized pneumonia was calculated using the relative risk (RR) obtained from the serotype distribution (pre and post PCV7 introduction) reported in 2 observational studies from Uruguay (calculated RR for PCV7: 0.73).We consider that this calculation is inappropriate since RR must be calculated based on incidence and the distribution of pneumococcal serotypes does not provide an equivalent parameter.
The authors also described that the pneumonia risk reduction of PCV10 and PCV13 was estimated based on the assumption that both would have the same DE as PCV7, plus an additional factor due to increased pneumococcal type coverage (calculated RR for PCV10: 0.48 & RR for PCV13: 0.17, respectively).The data sources used in the above mentioned calculations are not provided in the manuscript.The authors have also used a post-vaccination incidence estimated as 38%, 41%, and 17%, for the development of pneumonia with serotypes included in PCV7, PCV10, and PCV13, respectively.These later values were used as RR (0.38, 0.41 and 0.17) to calculate incidence reduction in page 1556, but they are not aligned with the previous RR calculations.Finally, Table 4 reports that the mean values of avoided hospitalizations were 31, 60 and 93 for PCV7, PCV10, and PCV13 respectively, and in addition the authors reported in the text that the probability of serotype isolation according to the decision tree would be 38%, 41%, and 17% for PCV7, PCV10, and PCV13, respectively.Overall, the explanation of these calculations are not detailed enough and we identify that the relative risk reduction or direct vaccine effect calculation was inappropriate since RR must be calculated based on incidence and the distribution of pneumococcal serotypes does not provide an equivalent parameter.

Pneumococcal types associated with pneumonia
The distribution of pneumococcal types in cases of invasive diseases in Peru (based on laboratory surveillance data) was used to calculate the incidence of clinical pneumonia associated to the pneumococcal types included in each vaccine.Although some of the pneumonia cases can lead to bacteraemia (~10%), most clinical pneumonia are local (mucosal) with no bacterial circulation in blood.Therefore, the pneumococcal type distribution observed in invasive disease does not necessarily represent the types most frequently associated with pneumonia.Furthermore, the true distribution of serotypes in pneumonia cases is unknown because it is not possible to obtain samples from diseased locations with present ethical standards.This is the main reason why clinical trials do not report serotype-specific vaccine efficacy against pneumonia.

Vaccine effect against pneumonia
We also looked for a potential association between the number of pneumococcal types included in the PCVs and their reported vaccine efficacy against pneumonia.We found no evidence in the scientific literature showing that expanded vaccine type coverage is associated with a greater vaccine efficacy against pneumonia [9][10][11][12][13][14][15].

Cost Effectiveness (CE) analysis
The incremental cost effectiveness ratio (ICER) calculation is based on the vaccination of 100 children with pneumonia.As the authors described in the text and Table 3, only the cost of a vaccination program comprising 100 infants with pneumonia was considered.Nevertheless, in order to obtain the health benefits reported over pneumonia prevention, it is necessary to vaccinate a much higher number of infants, than the 100 considered in the CE calculation.The method used in this study is well suited for an economic evaluation of a new medical therapy but not for preventive intervention like vaccines, where we need to target the intervention at a population level and not only to the diseased patients.This approach needs to be considered in the cost effectiveness model used.Finally, it is unclear how the numbers of averted hospitalizations (19, 35 and 77 avoided hospitalizations for PCV7, PCV10 and PCV13, respectively) reported in Figure 1 and 2 were computed.

Summary
In summary, the aim of developing a health economic evaluation of pneumococcal conjugated vaccines is valuable but needs to be developed with transparency and quality data.This is particularly important in health economic evaluations originating from Latin America, a region with insufficient tradition on Health Technology Assessments.An additional effort in detailing the methods used has to be emphasized so that decision makers and any interested professional of the region can better understand the analysis, its conclusions and limitations in detail.Only by considering the international recommendations, these economic evaluations can be a valuable tool to help informed decision making on vaccine introduction.Therefore, due to its limited perspective on vaccine benefits, limited information presented in the reported methods, and some inaccuracies introduced in the calculation of vaccine efficacy and the incremental cost effectiveness ratio, we believe that this study is insufficient for decision making on PCVs recommendation.