Uses of Research Evidence by State Legislators Who Prioritize Behavioral Health Issues

The purpose of research on mental illness and substance misuse (hereafter referred to as behavioral health [BH] research) is to improve population well-being. Public policy can help achieve this aim by allocating resources for evidence-based interventions and enacting regulations that support the prevention, management, and treatment of BH disorders (1–3). For BH research to inform policy decisions, however, policy makers must be knowledgeable about the research. Thus disseminating BH research to policy makers has been resoundingly acknowledged as a priority in the United States (4–10).

Dissemination is defined as the use of tailored strategies and preferred communication channels to spread research evidence among target audiences (11,12). The importance of policy-focused dissemination is apparent in the Substance Abuse and Mental Health Services Administration’s action plan (7), the National Institute of Mental Health’s (NIMH) objective to strengthen the public health impact of NIMH-supported research (8), the National Institute on Drug Abuse’s mission to ensure the rapid and effective dissemination of research results to inform policy (9), and the National Institute on Alcohol Abuse and Alcoholism’s mission of translating and disseminating research findings to policy makers (10).

Dissemination does not occur spontaneously, however, and the “passive” dissemination strategies typically used by researchers (for example, peer-reviewed publications) are generally ineffective at reaching policy makers (13–16). Dissemination is most effective when tailored to specific audiences (17), and the field of policy dissemination research has emerged to address this need by generating knowledge about policy makers’ preferences for receiving research evidence (for example, who delivers it and how it is presented) and the practices through which it is used (for example, when and where policy makers seek it) (18–20). Few policy dissemination studies have explored BH issues (21), however, and little empirical evidence exists to inform the design of policy-focused BH dissemination strategies.

A recent systematic review of interventions to increase the use of research in BH policy making highlighted the knowledge gap (22). The review identified only nine studies, and, as the authors noted, virtually none involved public policy makers (a policy maker was broadly defined as any decision maker, including program directors and community leaders). These results were consistent with previous reviews that identified few studies focused on how research evidence is used in BH policy making (23,24). Studies have explored uses of research in BH policy making in Canada (25,26), Australia (27,28), and Belgium (29), but they offer limited guidance regarding how to disseminate BH research to U.S. policy makers. Corrigan and Watson (30) extrapolated findings from social psychological research to propose factors (for example, political ideology) that might influence policy makers’ decisions about mental health services. The article, however, primarily drew from studies conducted with student populations, not with actual policy makers. A qualitative study of federal policy makers found that research evidence was important in passing BH parity legislation, but the study did not explore issues related to research dissemination (31).

BH research dissemination has been inadequately investigated among state policy makers. This knowledge gap warrants particular attention because most government authority to address BH issues exists at the state level. State policy makers, for example, determine the types of BH services that are reimbursable through Medicaid—the largest payer for BH services in the country (32,33). Federal legislation has attempted to promote BH insurance parity (34), but state legislation is needed to supplement federal laws and ensure equal coverage (35,36).

Our study addressed a critical gap in the field of BH services research by analyzing survey data collected from state legislators (that is, policy makers elected to the state houses or state senates). The primary aim of this exploratory study was to identify the research dissemination preferences and research-seeking practices of legislators who prioritized BH issues and describe the role research plays in determining their policy priorities. The secondary aim was to assess whether legislators who prioritize BH issues differ from legislators who do not prioritize these issues.

Theoretical Framework

Our decision to focus on legislators who prioritize BH issues was motivated by Kingdon’s (37,38) multiple streams theory of the policy-making process. The theory is founded on the premise that countless issues are constantly competing for policy makers’ attention and that three “streams” determine whether and how issues are addressed: a problem stream, consisting of issues that need to be addressed; a policy stream, consisting of solutions to these issues; and a political stream, consisting of public opinion and the broader sociopolitical environment. When these three streams converge around an issue, a “policy window” opens and “policy entrepreneurs” (that is, people who advocate for a specific issue) can advance their proposals to address it.

Accordingly, our study focused on legislators who perceived BH issues as priorities because these legislators are most likely to act as policy entrepreneurs and advance policies to address BH issues when a policy window opens (39). By producing a better understanding of how this important subgroup of legislators uses research evidence, the study can inform how BH research might be most effectively communicated to them via targeted dissemination strategies. By assessing whether this subgroup of legislators differs from legislators who do not prioritize BH issues, the study can signal whether tailored dissemination strategies are needed or if uniform strategies are sufficient.

Methods

Design, Participants, and Recruitment

We conducted a cross-sectional survey of state legislators. We partnered with the National Conference of State Legislatures to obtain a complete list of state legislators (N=7,525) and generated a random sample of 2,000 legislators who were invited to participate in a telephone-based survey between January and October 2012. Up to ten attempts were made to contact each legislator, 1,719 were successfully contacted, and 862 completed the survey (response rate, 46%). This is considered a good response rate for a legislator population (40). The survey was administered by an independent research firm. The survey was anonymous; answers cannot be traced back to particular respondents. Institutional review board approval was obtained from Washington University in St. Louis.

Measures

We measured legislators’ use of research evidence with an instrument that has been validated with state legislators (16,41,42). Details about the sources of instrument items and reliability coefficients are described elsewhere (16).

Legislators’ prioritization of BH issues.

Whether or not legislators perceived BH issues as policy priorities (yes or no) served as the dependent variable. Two items were used to determine this. First, legislators were prompted with the open-ended question: “Describe one or two policy priorities that are most important to you” (Q1). Legislators were then asked to select “the top three most important health issues for policy action in your state” (Q2) and presented with a list of 19 issues: “mental health,” “prescription drug abuse,” “access to healthcare,” “aging,” “cancer,” “diabetes,” “diet/nutrition,” “heart disease,” “HIV/AIDS,” “infectious diseases,” “injury prevention,” “Medicare/Medicaid,” “obesity,” “physical activity,” “quality of healthcare,” “the environment,” “tobacco use/cessation,” “universal coverage,” and “violence prevention,” in addition to an open-ended “other” option. At least one “other” health issue was named by 77% (N=664) of respondents.

Open-ended responses from Q1 and Q2 were searched for the terms “mental,” “behavioral,” and “psych” to screen for mental health issue prioritization and for “substance,” “drug,” and “alcohol” to screen for substance misuse issue prioritization. Each response was coded as “mental health,” “substance misuse,” “both,” or “neither.’’ A legislator was categorized as perceiving BH issues as priorities if he or she identified a mental health issue, a substance misuse issue, or both in open-ended responses to Q1 or Q2 or selected “mental health” or “prescription drug abuse” in response to Q2.

Factors that determine legislators’ health policy priorities.

As a follow-up to Q2, legislators were asked to select “which two factors most help you determine which health issues you work on” and were presented with a list of seven options.

Legislators’ research dissemination preferences.

Legislators were presented with 12 statements about features of disseminated research and instructed to “rate the level of priority that you attribute to each” on a scale of 1, low priority, to 5, high priority. To assess how perceptions of disseminated research varied according to the source, legislators were also presented with a list of eight sources of disseminated research (for example, universities and the media) and instructed to “rate the reliability” on a scale of 1, very unreliable, to 5, very reliable.

Legislators’ research-seeking and utilization practices.

Legislators were presented with 11 statements about research-seeking and utilization practices. Seven statements focused on where legislators go when seeking research evidence, and four focused on what they do with it. For each statement, legislators were prompted with the statement, “When making policy, how often do you . . .,” and they rated each on a scale of 1, never, to 5, always.

Legislators’ individual characteristics.

We collected information on legislators’ gender, educational attainment, parental status, self-rated health status, number of years served in the state legislature, chamber (house or senate), and political party membership. We also asked legislators whether they identified as liberal, moderate, or conservative on social issues and fiscal issues.

Analysis

Univariate descriptive statistics were produced to characterize the study population, stratified by whether or not the legislator prioritized BH issues. For dichotomous independent variables, Pearson chi-square and Fisher’s exact tests were used to identify associations with BH issue prioritization. Unadjusted odds ratios (ORs) with 95% confidence intervals (CIs) were produced to quantify the magnitude of these associations. For continuous and ordinal independent variables, we conducted two-tailed, independent-sample t tests and Mann-Whitney U tests to compare means between legislators who did and did not prioritize BH issues. Responses of “don’t know” and “refuse to answer” were coded as missing and excluded from analyses. Missing data did not exceed 3% for any variable.

Results

Of the 862 legislators who completed the survey, 125 (15%) identified BH issues as priorities (Table 1). Of these, 60 (48%) identified mental health issues, 57 (46%) identified substance misuse issues, and eight (6%) identified both. There were no significant differences between legislators who identified mental health issues and legislators who identified substance misuse issues. Legislators who prioritized BH issues were not significantly different from those who did not prioritize BH issues in terms of any individual characteristic.

Factors Determining Health Policy Priorities

Legislators who prioritized BH issues were significantly more likely than legislators who did not to select research evidence as one of the two factors that determined health policy priorities (32% versus 20%, OR=1.91, p=.002) (Table 2). Research evidence was the second most frequently cited factor by legislators who prioritized BH issues, whereas it was fifth most frequently cited by legislators who did not prioritize BH issues. Constituents’ needs and opinions was cited as the primary factor that determined health policy priorities by legislators who prioritized BH issues (66%) and by those who did not (68%). Among both groups of legislators, economic issues (N=151, 18%) and interactions with lobbyists (N=62, 7%) were the factors least frequently cited.

Research Dissemination Preferences

In addition to being more influenced by research when determining health policy priorities, legislators who prioritized BH issues were more discerning consumers of research evidence than legislators who did not prioritize BH issues. Legislators who prioritized BH issues gave a higher importance rating than legislators who did not to ten of 12 statements about features of disseminated research (Table 3). Differences were statistically significant for four of these features: unbiased (p=.014); presented in a brief, concise way (p=.044); delivered by a person the legislator knows and respects (p=.033); and telling a story about how a health issue affects constituents (p=.030). The unbiased aspect of research was the factor with the highest rating among legislators who prioritized BH issues, whereas it was tied for fourth among legislators who did not prioritize BH issues. The political feasibility of research received the lowest rating regardless of whether BH issues were identified as priorities.

Legislators who prioritized BH issues generally perceived research to be more reliable than legislators who did not prioritize BH issues, regardless of the dissemination source (Table 3). Legislators who prioritized BH issues reported a higher reliability rating than legislators who did not prioritize BH issues for seven of eight sources of disseminated research, although none of the differences were significant. Universities were identified as the most reliable source and media as the least reliable source by legislators who did and who did not prioritize BH issues.

Research-Seeking and Utilization Practices

Legislators who prioritized BH issues reported being more active seekers and users of research than legislators who did not prioritize BH issues. The former group engaged in eight of 11 research-seeking and utilization practices more often than those who did not prioritize BH issues (Table 4), although the only significant difference was for the practice of attending seminars or presentations where research was discussed (p=.012). Among all legislators, asking internal legislative research bureaus for information was performed most often and reading or watching popular media stories was performed least often.

Discussion

We found that legislators who prioritized BH issues actively sought and were influenced by research evidence—and more so than legislators who did not prioritize BH issues. These results suggest that the legislators who are most likely to act as BH policy entrepreneurs and best positioned to integrate BH research findings into policy designs are particularly receptive to research evidence and have distinct dissemination presences, although the magnitude of these differences was small. Future studies should examine in greater detail how BH research is used in state legislative processes and test the effects of legislator-focused BH research dissemination strategies. Our results offer guidance to structure the design of these strategies.

The finding that legislators who prioritized BH issues reported a significantly higher importance rating for the statement that research “tells a story of how a health issue affects my constituents” suggests that dissemination strategies that combine narrative with local BH data could be effective. Studies have found that narrative approaches to policy dissemination, in which stories about individuals affected by an issue are presented, are often more effective than exclusively data-focused approaches (43–45). This is consistent with research finding that people’s perceptions of mental illness improve when they know a person with a diagnosis (46) and Corrigan and Watson’s (30) proposition that mental health policy advocacy is likely to be most effective when it involves personal stories about mental illness.

The finding that legislators who prioritized BH issues significantly preferred research delivered by a known and trusted individual suggests potential for initiatives that foster relationships between BH researchers and legislators. An initiative to enhance collaboration between mental health researchers and policy makers in Canada offers a model (47). BH researchers might also establish relationships with, and disseminate research findings directly to, legislative research bureau staff, given that these staff were identified as legislators’ primary source for research evidence.

Our study can inform how BH researchers might utilize the media to infuse research evidence into policy-making processes. Scholars have asserted that the media play a major role in shaping BH policy (48,49). We found, however, that legislators rated the media as the least reliable and least utilized source for research. It is possible that BH research influences policy making indirectly via legislators’ constituents, who become knowledgeable about BH research through the media and advocate for legislators to act on this research. As noted, constituent needs and opinions were identified as the most influential factors determining health policy priorities, and constituents were perceived as the second most reliable source of research.

We did not observe significant differences in political ideology between legislators who did and did not prioritize BH issues. This suggests bipartisan support for BH policies, a situation that would increase the chances of BH policy proposals’ becoming law. However, our study did not explore legislators’ opinions about the types of policy approaches that should be used to address BH issues or the extent to which they were evidence based or varied by political ideology. As Corrigan and colleagues (50) described, conservative and liberal legislators are likely to consider decisions to allocate resources for mental health services differently, with liberals being more inclined to support government funding of services.

Limitations of this study derive from the fact that we conducted a secondary analysis of an existing data set. Measures assessed legislators’ preferences and practices related to research evidence in general, not BH research in particular. This limits what can be inferred from the study. For example, we observed a strong association between identification of research evidence as a factor that determined health policy priorities and legislator prioritization of BH issues, but we cannot infer that BH research evidence contributed to legislators’ determination that BH issues were a priority.

There are also limitations with our broad measure of BH issue prioritization. Legislators identified a range of specific BH issues (for example, depression among elderly people and prescription drug abuse among adolescents) in open-ended responses, but we combined these issues into a single measure because we did not have statistical power to conduct subissue analyses. Although many BH conditions are comorbid and simultaneously addressed by a single law (for example, state BH parity legislation), they affect distinct populations with specific needs. Future research should explore the particular types of research used by legislators when they are working on specific BH issues.

There are similar limitations with our measure of non–BH issue prioritization. Legislators in this group identified a various issues as policy priorities in response to Q1, some of which are not the subject of extensive research (for example, unemployment and transportation). Consequently, compared with legislators who prioritized BH issues, legislators who did not may have reported engaging in research-seeking and utilization practices less frequently because they were focused on issues for which research evidence is less relevant. This is unlikely to affect the implications of our study, however, because it simply signals that legislators who prioritize BH issues may engage in different research-seeking practices than legislators who do not prioritize these issues and that tailored dissemination strategies may be warranted. Our study was designed to assess whether use of research practices differed between legislators who did and did not prioritize BH issues, not to determine the reasons for the differences observed.

Conclusions

Compared with legislators who did not prioritize BH issues, legislators who prioritized BH issues more actively sought, had distinct preferences for, and were more influenced by research evidence. These results suggest potential for BH research findings to inform legislative decisions. Developing and testing policy-focused BH research dissemination strategies are areas for future research that can advance evidence-based policies to improve the well-being of people affected by BH issues.