SACPA: California’s Substance Abuse and Crime Prevention Act; SUD: Substance Use Disorder; CJS: Criminal Justice System; DID: Difference-in-Differences

Offenders with Substance Use Disorders (SUDs) have long been an issue throughout the Criminal Justice System (CJS). Due to such factors as mandatory minimum sentences, three-strike laws, and other “tough on crime” policies and practices, the rate of incarceration for nonviolent drug offenders has produced unprecedented incarceration rates over the last few decades [1]. To address these issues, many diversion and treatment efforts have been designed, implemented, and evaluated [2]. Evaluation results have generally been positive [3,4]. At the same time, the number of offenders with substance use problems who receive treatment is low; according to the Bureau of Justice Statistics [5], in 2004 only about 15% of state prisoners who met criteria for drug dependence or abuse had participated in a drug treatment program with a trained professional since admission. Although drug courts have been popular as a way to divert drug-involved offenders from jail or prison to community treatment, there is doubt that they can serve a sufficiently large enough population to substantially reduce the jail and prison populations [6].

Policy initiatives and their effects are dependent, in large part, on the environments within which they are implemented. California’s voter-initiated Substance Abuse and Crime Prevention Act (SACPA, commonly known as “Proposition 36”) is a statewide offender diversion program that exemplifies national efforts to redirect drug-using offenders from the criminal justice system into the SUD treatment system. Under SACPA, adults convicted of nonviolent drug possession offenses can receive community-based drug treatment in lieu of traditional sentencing. Probationers or parolees who violate drug-related conditions of community supervision are also eligible. In operation since 2001, SACPA implementation successfully applied a public health approach to drug-related offending, reducing drug use among offenders [7,8]. Many drug-abusing offenders entered treatment within a very short time period [9], and the program resulted in favorable outcomes, especially among treatment completers [10], while yielding noteworthy cost savings [11-13]. However, most of the research on SACPA has focused on evaluating offender-level outcomes. How particular social contexts impose opportunities and constraints in ways that impact the performance of the SACPA program, especially in terms of cost effects, has received relatively little attention. Ecological models have been used to examine the impact of a variety of health behaviors [14-17], but this approach has been little applied to achieve a better understanding of SUDs and recovery from SUDs.

Contextual variation

SACPA is a statewide program, but its implementation was determined by each of the 58 counties in California through policies and procedures that were tailored to local conditions [18]. Jurisdictional, or county-level, variation has been reported in SACPA program operations [8,18-25], characteristics of participating offenders [18], and outcomes [26].

The attitudes, training, and practices of key implementers impact the degree of success, or failure, of social policy initiatives [27], as does the nature of the linkages between cooperating agencies (i.e., exchanges of communication, trust, and resources) and the boundaries that exist among those key actors [28,29]. Policymaking is not necessarily centrally planned or controlled but is rather the product of complex, dynamic, and interdependent processes. Actors from different organizations may form alliances that differ in their effects, intended and otherwise [30]. Relationships may be further influenced by institutional settings [31,32], and they evolve over time. Similar processes have been identified as being salient within the SACPA environment [11,19,24,33]. In particular, the level of collaboration between SACPA’s key stakeholders has been identified as a significant contextual element that greatly contributed to the program’s success [19,24].

Economic variation

In our prior work [11], we assessed the overall cost effects of SACPA. A major finding was that the SACPA program saved government $2,317 per eligible offender. The majority of savings was due to large decreases in incarceration costs moderated by smaller increases in re-arrest and SUD treatment costs. Other work examined specific costs and savings associated with variability in offender demographic characteristics (gender, race/ethnicity, and age) and functioning (criminal, mental health, and employment). A significant gender difference was revealed, in that the program was more effective in reducing offender costs for men than it was for women [11]. SACPA-associated savings were also found to be moderated by offenders who had an extensive criminal history and those who had a documented history of mental illness [34].

In the current paper, we extend findings to county-level factors that significantly affect SACPA cost variation, focusing specifically on the population served and the level of collaboration between relevant stakeholders. We hypothesized that those counties serving more severe offender populations would see lower SACPA-associated savings. Additionally, we hypothesized that SACPA-associated savings in counties experiencing less collaborative stakeholder interactions would also be lower.

Of the available 88,962 SACPA-eligible offenders, 80,627 (90.6%) had complete data and were used for analysis. No significant differences were identified between the full population cohort and the completedata subsample. Most of the sample was male (76%), mean age was 34.2 years (SD=9.2), and race/ethnicity was 46.9% African American, 30.0% Hispanic, and 16.5% non-Hispanic White.

Two groups were constructed using a time-lagged cohort approach with individuals meeting SACPA-eligibility criteria before the program was enacted, the “pre-SACPA cohort” (July 1, 1997, to June 30, 1998, n=47,355) being compared to the “SACPA cohort,” which was composed of SACPA-eligible offenders convicted within the first 12 months of program implementation (July 1, 2001, to June 30, 2002, n=41,607). Both population-level cohorts were identified through the California Department of Justice (DOJ) database and were followed on an intent-to-treat basis; that is, all cohort members were included whether or not they participated in SACPA or entered treatment. We applied a difference-in-differences analysis, which is a commonly used and empirically rigorous technique for the evaluation of impacts of policy changes [35]. Data on health-care utilization, criminal justice involvement, and SUD treatment participation were obtained from statewide administrative databases for 30-month periods before and after the identifying conviction, constituting a total of 60 months of offender observation. The study was approved and monitored by the UCLA Institutional Review Board and the California State Human Subjects Protection Committee.

Data sources

Statewide offender-level administrative data were obtained from four sources. Arrests and convictions were retrieved from the California Department of Justice Automated Criminal History System. Substance use disorder treatment admissions and discharges were captured in the California Department of Alcohol and Drug Programs California Alcohol and Drug Data System (CADDS). Prison and parole records were captured in the Offender Based Information System (OBIS), maintained by the California Department of Corrections and Rehabilitation. Health resource utilization was captured in the Medi- Cal (California Medicaid) claims data, as recorded by the Department of Health Care Services. Individual-level data were linked through the use of name, birth date, social security number, and other identifying information. Quality control assessments showed a high degree of successful linkages. An annual Stakeholder Survey provided data on implementation issues from the perspective of SACPA stakeholders. Questions regarding communication, formal and informal linkages, and information sharing were assessed based on previous work showing that agency collaboration contributes to SACPA outcome success in SUD treatment. Of the 58 California counties, 38 provided the utilized county-level data and were included in the analysis.

Primary outcome variables

Statewide offender-level administrative data were used to calculate the total cost-differences across eight domains (i.e., prison, jail, probation, parole, arrests, convictions, publicly funded healthcare utilization, and SUD treatment) between the 30 months before and after a SACPA-eligible conviction [11]. Costs are presented in 2009 U.S. dollars.

County-level predictors

As noted earlier, level of stakeholder collaboration was developed from an annual stakeholder survey that contained items on implementation issues from the perspective of SACPA participant agencies. Questions regarding communication, formal and informal linkages, and information sharing were assessed.

Severity of SACPA population was constructed as an index score for each county. Variables for the index were selected based on their previously documented relationship to SACPA outcomes [11,36,37]. Poorer SACPA outcomes have been observed among parolees, racial/ ethnic minorities, men, users of drugs other than methamphetamine (particularly cocaine), and offenders who are unemployed. Administrative data on the characteristics of offenders at entry into drug treatment during 2006-2007 were used to create the index. The index was created as a count of the number of categories in which a specific county had SACPA-eligible offender populations that were one standard deviation or higher than the California state average on the earlier mentioned characteristics that have been associated with poorer outcomes. Index scores ranged from -1 to 3, with a higher score indicating and were centered at the median (i.e., -1).

Covariates

To account for possible trends affecting the time-lagged control sample, county-level indicators of crime were collected by calendar year for each county from publicly available data from the office of the California Attorney General (www.ag.ca.gov).

Statistical analysis

Multi-level, multivariate, linear difference-in-differences regression models were estimated on the total cost difference for the sample as a whole. The analysis utilized the difference in post- versus pre-conviction costs for SAPCA and pre-SACPA cohort offenders by subtracting the between-cohort differences in these pre-post differences (i.e., the difference-in-differences [DID]) using a SACPA indicator included in the regression equation. Additional details of this method are described in Anglin et al. [11]. Offender-level covariates included race, gender, age, and SACPA participation. County-level covariates included level of SACPA stakeholder collaboration difficulties, severity of the SACPA population, and county per-capita crime levels.

Fixed effects were estimated for each predictor, whereas countyspecific estimates, over and above individual level SACPA effects, were estimated and reported for each of the included counties. The final estimated multi-level model was specified as (Table 1):

Table 1: Multilevel SACPA Difference-in-Differences cost-effect (38 counties, N = 80,627 offenders)

Given county variability in the overall SACPA-slope term (expressed as β_1j), it was examined as a measure of county-specific SACPA-participation total-cost effect. To calculate the overall effect of SACPA for each county with a significant SACPA county-specific effect (μ_1j), the fixed SACPA effect (γ₁₀), county-specific contribution of the population-index term (γ₁₀ Cindex), agency communication (γ₁₁ Agency-Communication), county per capita crime (γ₁₂ County Crime), and the county SACPA county-specific effect were summed when significant. All statistical analyses were performed using SAS version 9.1.

County variability in SACPA savings

The distribution of the overall SACPA effect is shown in Figure 1. Cost savings were associated with SACPA participation for 36 of the 38 counties that were analyzed. Counties produced SACPA cost benefits of approximately $4,100 per eligible offender after controlling for offender-level variability as well as variations in the severity of the population served and statistically different (above or below) from the overall individual-level fixed effect of $3,190 in savings per SACPA eligible offender (see regression estimates in Table 1). For illustrative purposes, Table 2 displays the county-level summary characteristics of these 11 counties. Given the potentially sensitive nature of the implementation issues discussed and their possible implications on funding allocation for SUD treatment in California, counties are identified by a number and not by name. As shown, there is substantial variability in the characteristics of SACPA-eligible offenders by county.

Table 2: Characteristics of counties with significant SACPA county-specific effects

Figure 1: Distribution of the total SACPA cost effect (n=38 counties).

The total and component cost estimates for these 11 counties are displayed in Figure 2. Only significant county-level covariates, namely the county SACPA-population index (β=-713, p<.05) and the stakeholder communication problem rating (β=-998, p<.05), were included in the calculation of the county-specific SACPA effect. Results indicate that a more severe SACPA population and less collaboration among stakeholders resulted in higher cost estimates. What would happen to our cost estimates if we were to adjust them by accounting for these county-level factors? We provide two examples of adjusted calculations, one being for the least cost-effective county (#9) and the other for the most cost-effective county (#11).

Figure 2: Total SACPA effect in counties with effects significantly different from the mean (n=11 counties)

County #9, with approximately 900 participants, yielded a SACPA effect of -$3,237. The population served by this county was assessed to be more challenging than the median county population served, with a county-population index equal to 1. Accordingly, for comparative purposes, we adjusted this county’s cost-estimate downward by subtracting $713 from their overall cost estimate, resulting in an adjusted estimate of -$3,950. Additionally, this county’s level of stakeholder collaboration was low, resulting in a further adjustment by -$998, yielding a revised estimated total of -$4,948. Finally, we added the fixed SACPA effect of -$3,190, resulting in a final estimate of -$8,138. In other words, if County #9 had had an average SACPA population and stakeholder communication level, we estimate it would have achieved savings of $8,138 per eligible SACPA offender.

By contrast, County #11, with approximately 1,500 participants, yielded a much higher than average county-specific SACPA effect of $6,440. Moreover, the population served by this county was rated as being less challenging (i.e., -1), resulting in an upward adjustment by $713; thus resulting in an adjusted estimate of $7,153. Additionally, inter-agency communication was deemed to present no problems as indicated by a score of 0, resulting in no adjustment. Finally, the fixed SACPA effect of -$3,190 was added, resulting in the final estimate of $3,963 for County #11. In other words, after taking into account the benefit that County #11 had from a less challenging population, if it had served an average population, it would have had an additional cost of $3,963 per eligible SACPA offender. (It should be noted that these calculations do not reflect the final per-offender cost in these counties but, rather, only the effect of SACPA participation and its relationship to county variability in the population served and the reported communication effectiveness.)

The goals of the present analysis were, for comparative purposes, to examine the county-specific variability in the effectiveness of SACPA, a state-wide diversion program implemented individually by each of California’s 58 counties. In previous work, SACPA was found to produce a net cost-savings of $2,317 when examined collectively, or statewide [11]. The present analysis replicates the previous findings regarding SACPA’s overall cost-effectiveness, but indicates that county-specific findings significantly diverge in terms of SACPAassociated cost savings.

However, given the substantial variation in SACPA-associated savings based on demographic factors such as race, gender, age, and functioning (i.e., offending history, mental health diagnosis, and employment) noted in other analyses of this data, it is not unexpected that controlling for these factors at the county level produced variations in the estimated SACPA cost effect. Indeed, variations in the SACPA population served, as well as in the effectiveness of communication between relevant county agencies, produced a broad range of SACPA effects, from a savings of $8,138 per offender to an increased cost of $3,163 per offender after adjustment for individual demographics and county-level variability. This finding fits well with previously published work revealing differences in SUD treatment outcomes such as recidivism and abstinence between different counties, as well as with previous work on differences in the level of inter-agency collaboration and the impact of such differences on program effectiveness [19].

Of the 11 counties producing significant deviation from the overall SACPA effect of $3,190 in savings, 9 counties produced overall savings, with 5 of these counties producing substantially greater savings than previous analyses revealed and 4 others producing more modest savings. Of the two counties that showed increases in cost, only one revealed substantial increases in costs ($2,965 per offender). This suggests that the cost-benefits of SACPA were relatively robust, even in the face of difficult populations and of suboptimal inter-agency collaboration environments.

It should be noted that our analysis utilized data from the first year of SACPA implementation. We found no absolute pattern of results between county-specific SACPA effect and population severity or communication effectiveness, meaning that counties that served less compromised populations did not necessarily produce greater savings, as seen in our example calculation using County #11. It is possible that counties that were better prepared to deal with the relatively large influx of offender clients experienced during the initial year of SACPA produced greater savings than did counties that were caught less prepared. Future analysis should expand the methods used here to assess for improvements in county-specific SACPA cost-benefits in later years of the program.

Potential policy and practice implications

SACPA was meant to reduce the state’s inmate census and incarceration costs while providing an opportunity for SUD treatment for this costly population. The assessment of variations in program effectiveness, as well as costs related to contextual factors, can help to refine and clarify the specific and desired outcomes of the program. Model implementation strategies, both negative and positive exemplars, can be identified and their differential effects assessed. Moreover, defining such cost variation can inform resource allocation to obtain the best “return on investment” for increasingly scarce funding, while identifying domains that require additional attention to adapt to jurisdictional parameters that optimize costs in the near and longer term. These methods may have application to other public policies, in that they could be used to assess the relative cost contributions of contextual factors to their implementation, especially as healthcare reform initiatives are currently under development nationally [38,39].

Potential methodological implications

Although the SACPA program was unique to California, the economic analyses reported here are widely applicable and are relatively easy to conduct. First, as noted in our data sources, a broad array of data that are collected on an ongoing basis for administrative purposes are available to various jurisdictions. While linking these data across domains is rarely straightforward, such data allow for the quantitative assessment of social policy changes and the evaluation of program effects [39]. Second, both broad and specific effects can be ascertained, allowing a better explication of the many influencing factors and conditions that affect cost estimates. Finally, results expressed in econometric terms allow for a better allocation of increasingly scarce governmental resources to plan for and optimize beneficial outcomes by, for example, improving selection criteria for participants, providing a set of interlocking and coordinated intervention elements, and tailoring program elements to meet specific client needs.

Study limitations

The present study has a number of limitations, including a lack of information about relevant county-level characteristics of SACPAeligible offenders who did not enter treatment. Primary drug of use as well as employment status were collected only at treatment entry and discharge, and since this information was used in determining the population-index used within the model, it is possible that the characterization of offenders within specific counties was biased. In order to assess for such bias, we used data from non-SACPA individuals who participated in publicly funded treatment during the same time period as our pre-SACPA and SACPA cohorts. We found no significant changes between the full sample and the utilized sample in terms of age, gender, and racial-makeup, strengthening our confidence in the appropriateness and lack of significant bias in our sample.

The county-specific manner in which SACPA was implemented resulted in wide variability in measures of program effectiveness and costs. Our findings suggest that a substantial portion of this variability can be attributed to differences in the local populations served and variability in the effectiveness of inter-agency communication within specific counties. While population severity and client characteristics are not necessarily aspects of programmatic implementation that can be controlled, SUD services attending to local population characteristics can be developed to meet county-specific needs.

More directly, it is certainly possible to obtain improvements in collaboration among responsible agencies. For this reason, funding agencies need to assure that adequate cost-effective procedures are in place to increase and facilitate such collaboration from the outset. Once such efforts are in place, programmatic effectiveness should be assessed with consideration of the population served. Moreover, program services that are designed to be responsive to demographic characteristics such as gender-specific services, employmentenhancement opportunities, and age-appropriate services should be considered for inclusion in overall program offerings. As healthcare reform initiatives integrating primary medical care and substance use disorder treatment are planned at local levels, their successful implementation will require consideration of such issues accordingly.

Thanks are due to Nicole Jamison-Dinowitz for her assistance in data preparation and to Bohdan Nosyk for editorial review, both of whom are part of UCLA Integrated Substance Abuse Programs. Funding was provided, in part, by the UCLA Center for Advancing Longitudinal Drug Abuse Research (CALDAR) under NIDA grant P30 DA016383. The authors thank the staff of CALDAR for statistical consultation. Adi Jaffe was supported under a postdoctoral fellowship funded through the Anglin Research Fund. This article is dedicated to Douglas Longshore, (deceased December 2005), who guided the SACPA evaluation during its first 6 years; Darren Urada carried the project forward.