Barriers to Generalizability of Health Economic Evaluations in Latin America and the Caribbean Region

Use and acceptance of health economic evaluations (HEEs) has been much greater in developed than in developing nations. Nevertheless, while developing countries lag behind in the development of HEE methods, they could benefit from the progress made in other countries and concentrate on ways in which existing methods can be used or would need to be modified to fulfill their specific needs. HEEs, as context-specific tools, are not easily generalizable from setting to setting. Existing studies regarding generalizability and transferability of HEEs have primarily been conducted in developed countries. Therefore, a legitimate question for policy makers in Latin America and the Caribbean region (LAC) is to what extent HEEs conducted in industrialized economies and in LAC are generalizable to LAC (trans-regional) and to other LAC countries (intra-regional), respectively.

We conducted a systematic review, searching the NHS Economic Evaluation Database (NHS EED), Office of Health Economics Health Economic Evaluation Database (HEED), LILACS (Latin America health bibliographic database) and NEVALAT (Latin American Network on HEE) to identify HEEs published between 1980 and 2004. We included individual patient- and model-based HEEs (cost-effectiveness, cost-utility, cost-benefit and cost-consequences analyses) that involved at least one LAC country. Data were extracted by three independent reviewers using a checklist validated by regional and international experts.

From 521 studies retrieved, 72 were full HEEs (39% randomized controlled trials [RCTs], 32% models, 17% non-randomized studies and 12% mixed trialmodeling approach). Over one-third of identified studies did not specifically report the type of HEE. Cost-effectiveness and cost-consequence analyses accounted for almost 80 % of the studies. The three Latin American countries with the highest participation in HEE studies were Brazil, Argentina and Mexico. While we found relatively good standards of reporting the study’s question, population, interventions, comparators and conclusions, the overall reporting was poor, and evidence of unfamiliarity with international guidelines was evident (i.e. absence of incremental analysis, of discounting long-term costs and effects). Analysis or description of place-to-place variability was infrequent. Of the 49 trial-based analyses, 43% were single centre, 33% multinational and 18% multicentre national. Main reporting problems included issues related to sample representativeness, data collection and data analysis. Of the 32 model-based studies (most commonly using epidemiological models), main problems included the inadequacy of search strategy, range selection for sensitivity analysis and theoretical justifications.

There are a number of issues associated with the reporting and methodology used in multinational and local HEE studies relevant for LAC that preclude the assessment of their generalizability and potential transferability. Although the quality of reporting and methodology used in model-based HEEs was somewhat higher than those from trial-based HEEs, economic evaluation methodology was usually weak and less developed than the analysis of clinical data. Improving these aspects in LAC HEE studies is paramount to maximizing their potential benefits such as increasing the generalizability/transferability of their results.