Skip to main content
  • ORIGINAL ARTICLE
  • Open access
  • Published:

Evaluating search and matching models using experimental data

Abstract

This paper introduces an innovative test of search and matching models using the exogenous variation available in experimental data. We take an off-the-shelf search model and calibrate it to data on the control group from a randomized social experiment. We then simulate a program group from a randomized experiment within the model. As a measure of the performance of the model, we compare the outcomes of the program groups from the model and from the randomized experiment. We illustrate our methodology using the Canadian Self-Sufficiency Project (SSP), a social experiment providing a time-limited earnings supplement for Income Assistance recipients who obtain full-time employment within a 12-month period. We find two features of the model are consistent with the experimental results: endogenous search intensity and exogenous job destruction. We find mixed evidence in support of the assumption of fixed hours of labor supply. Finally, we find a constant job destruction rate is not consistent with the experimental data in this context.

JEL Classification:J2; I38; J6

1 Introduction

Search and matching models in the spirit of McCall (1970) or Diamond (1982), Mortensen (1982) and Pissarides (1985) (DMP) are an important tool for the evaluation of new or existing labor market policies.1 The policy experiments conducted within such models allow consideration of the potential effects of policy reforms that have yet to be implemented in reality as well as the general equilibrium effects of large-scale reforms or of small-scale reforms that may be implemented on a large-scale in the future. They also allow what one might call theoretically structured mediation (or decomposition) analysis, wherein the model aids in empirically sorting out the causal channels underlying the “black box” impact estimates that emerge from design-based studies. Despite the importance of this policy evaluation tool, it is difficult to put a high degree of confidence in the quantitative results of such exercises, as it is often difficult to judge how well the model captures the responses of individuals to changes in government policy.2

In general, the literature evaluates the performance of structural economic models, such as those in the search and matching literature, in two ways. The most common way considers the in-sample fit of the model relative to the moments used in the calibration or estimation. In many cases, the ability of the model to match moments not included in the calibration or estimation process, but drawn from the same empirical context, is also used as a test of the model.3 This sort of test represents an important starting point to ensure the benchmark model can replicate behavior observed in the data used to calibrate or estimate it. However, such tests provide no direct evidence on how reliable the model predictions are under changes to the policy environment. The second way to gauge a model’s performance is out-of-sample tests, where the fit to other time periods (or other empirical contexts more generally) is used as a test of the model. This is a more demanding measure of performance, as the model’s predictions are compared to an environment outside of the calibration or estimation process. However, because many features of the economy vary over time, it is difficult to gauge the extent to which the comparison between out-of-sample statistics and the model predictions are contaminated by changes in factors not captured by the model.4

The contribution of our paper is twofold. First, we introduce a new and convincing test of the quantitative performance of search and matching models. The new test that we consider exploits the wealth of information available through social experiments (and readily generalizes to the broader set of compelling non-experimental impact analyses). In social experiments, small subsets of the population are randomly assigned to program and control groups; the program group is subjected to a potential policy reform, and the difference in outcomes between the groups provides an estimate of the mean impact of the policy.5 Social experiments provide compelling causal estimates of how individuals within the experiment respond to the incentives introduced by the policy reform. It is in this sense that social experiments provide an excellent opportunity to determine whether a particular model, once calibrated or estimated, can accurately predict the effects of policy changes.6

The basic idea behind our approach is the following: First, we calibrate our model of interest to match the behavior of the control group from a randomized experiment. Second, we introduce the policy of interest in the model and use the calibrated model to simulate a program group. Third, we compare the outcomes of the model program group with the outcomes of the experimental program group. The model is asked to match the outcomes of the program group without relying on the exogenous variation introduced within the experiment. Further, as in the model, the only change in the economic environment that is introduced in the experiment is the policy change of interest. Therefore, we obtain a test of the model uncontaminated by changes in other factors that could blur the comparison between the model and the data. In this respect, the social experiment provides a very rigorous test of the model.7

The second contribution of our paper is to determine whether the model is an appropriate tool to use in estimating the behavioral response of the Canadian Self-Sufficiency Project, a small-scale experiment designed to provide incentives for individuals on Income Assistance (IA), the Canadian welfare program, to leave the system and seek employment.8 We use the experimental data on single mothers from the SSP to assess a textbook model of the labor market, as in, e.g., Pissarides (2000) or the recent survey by Rogerson et al. (2006). The SSP provides financial incentives by offering temporary earnings supplements to individuals on IA. Individuals must remain on IA twelve months to become eligible for earnings supplements; once they do, they receive a supplement if they become employed and leave Income Assistance within the following twelve months. The model is augmented to incorporate the main features of the SSP. In particular, the model allows for time limits in determining eligibility for receipt of the supplement, consistent with the one-year time limit in the experiment, allows individuals to receive the earnings supplement for up to three years while employed, and allows the earnings supplement to depend on the earnings received by eligible recipients. The time limitations for entry to and exit from the Canadian unemployment insurance program (called Employment Insurance or EI) and the interactions between IA, the unemployment program, and the labor market are also incorporated in the model.

The main margin through which the SSP is designed to affect behavior is via increased job search effort by welfare recipients resulting from the time limitations and the financial incentives introduced by the program. Therefore, as in Pissarides (2000, Ch.5), we also introduce endogenous search intensity in the model. Because of the policy focus on search effort we do not include a reservation wage channel in the model. Our paper focuses on testing the partial equilibrium implications of the model, as the SSP is a small-scale experiment that is unlikely to have general equilibrium implications for the labor market.

After constructing the model, we calibrate the model in the absence of the program using publicly available, non-experimental data on single mothers and data on the experimental control group. The SSP was implemented separately in two provinces, New Brunswick and British Columbia, and we calibrate and test the model separately for each province. The parameters calibrated in the first stage include the discount factor, search friction parameters, and exogenous job separation rates: parameters that are, in theory, invariant to changes in the Income Assistance program. We then simulate the SSP experiment within our calibrated model and compare the model outcomes for the simulated control group, the simulated program group, and the difference between them (i.e., the simulated experimental impact) to those in the experimental data.

Our tests focus on three features of the model: search intensity, job destruction, and earnings. First, we consider how well the model specification for search intensity matches the experimental data. The parameters governing the choice of search intensity cannot be directly identified in the data and are therefore the margin that we have the least confidence in. We test the search intensity features of the model by comparing the exit rates from IA for the control and program groups in the model to those in the data. In British Columbia, we find that the impacts of the SSP on the IA-to-work transition rates during the 53 months following random assignment in the model are not statistically different from those in the experimental data. We are also able to match the delayed-exit effects of a second randomized experiment in British Columbia that offered SSP to new Income Assistance recipients. Along these dimensions, our model test provides strong support for the framework. In contrast, the model predicts a significantly (and substantially) higher transition rate from IA to employment with the introduction of the SSP in New Brunswick. The elasticity of search costs with respect to search effort in New Brunswick must be much higher than that in British Columbia to match the experimental impact of the SSP.

The second feature of the model we consider is the constant, exogenous job destruction rate. The model test produces two findings. First, we find that the data do not reject an exogenous job destruction rate for the program and control groups in either province, as there are no statistically significant differences in the employment survival rates for the program and control groups, even when individuals in the program group stop receiving supplement payments. However, the assumption of a constant job destruction rate is not consistent with the experimental data, as the employment exit hazard rate clearly falls with tenure.

We also assess the effects of the policy change on earnings. In partial equilibrium, we expect no change in earnings, conditional on tenure, due to the small number of individuals affected by the policy change in the experiment. For British Columbia, this appears consistent with the data as the earnings-tenure profiles for the employed in the program group do not statistically differ from those for the employed in the control group.9 However, although the earnings-tenure profiles for the employed in the program and control groups are the same, a closer examination of the data indicates that hours are higher in both provinces for those individuals eligible for the SSP, consistent with the program’s focus on full-time work.

In the final component of our analysis, we switch from using the experimental program group data to test the model to using it to help calibrate the model. This switch embodies in our context the very general tradeoff between using additional restrictions for testing versus using them for calibration or estimation. For example, in a common effects linear instrumental variables context, a researcher with two candidate instrumental variables must choose between using both for estimation, or using one to test the exogeneity of the other. A natural approach within our framework starts with using the experimental variation to test the model and then switches to using it for estimation or calibration once a satisfactory model is found. In the SSP context, using the experimental program group in the calibration changes the results in New Brunswick but not in British Columbia.

We organize the remainder of the paper as follows: Section 2 presents a prototypical matching model with endogenous search intensity. Section 3 outlines our methodology for testing the model and presents data on the social experiment used to test the model. A comparison of the model experiment and social experiment, including formal model tests, is presented in Section 4, along with sensitivity analysis and the analysis that uses the experimental variation for calibration rather than for testing. Section 5 concludes.

2 The model

In this section, we present the model of the labor market that we use to conduct equilibrium program evaluations. Three segments of the market are incorporated in the model: individuals may be employed (E), unemployed and receiving unemployment benefits (U) or on Income Assistance (A).10 This feature of the model allows us to consider how currently employed workers, unemployed individuals and Income Assistance recipients interact in the labor market. The model builds on the standard DMP framework where individuals maximize expected lifetime income by choosing their labor market state and the intensity with which they search for work if not employed. The model is extended to incorporate the details of the Income Assistance program and time limitations for entry to and exit from the unemployment insurance program. We focus here on the problem faced by workers and abstract from the wage determination process and the model equilibrium.11 The reason for this abstraction is that we are only able to evaluate the performance of the model in partial equilibrium: the social experiment we use to test the model affects only a small number of individuals and as such is not expected to have any effects on the distribution of wages in the economy.

Key features of typical unemployment insurance and Income Assistance programs are incorporated in the model as follows. First, individuals face time limitations regarding entry to and exit from the unemployment system. Individuals who enter employment from Income Assistance or who have exhausted their unemployment benefits become eligible to receive unemployment benefits after I months of employment. The number of benefit months subsequently increases by one month for each additional month of employment, from a minimum of \(\underline {u}\) months up to a maximum of \(\bar {u}\) months. Workers who enter employment with unused benefits retain their unused benefit months and accumulate additional months with each month worked. Second, individuals who exhaust their unemployment benefits and do not secure a job are assumed to transit directly to Income Assistance. Finally, it is assumed that individuals can remain on Income Assistance indefinitely or transit to employment if they contact a firm with a vacancy; Income Assistance recipients cannot transit directly from IA to unemployment. In the following sections, we describe the problems faced by individuals in each labor force state in the model.

2.1 IA Recipients

IA recipients receive benefits (b a ) and pay search costs c a [p(0)z] every month they remain on IA, where z is the elasticity of search costs with respect to search effort, c a is a parameter capturing the disutility of search effort, and p(i) is optimal search effort for individuals with i months of unemployment benefits remaining. The cost of search depends directly on the intensity with which individuals search within the model. In particular, for values of z>1, the marginal cost of search increases as search effort increases. If IA recipients contact a firm with a vacancy, they transit to employment. Otherwise, they remain on IA in the next period. The value function for an IA recipient is

$$V^{A}= \max_{p(0)} \left\{b_{a}-c_{a}[p(0)^{z}]+ \beta \left[m(0)V^{E}(1,0)+(1-m(0))V^{A}\right] \right\}, $$

where m(0) is the match rate for IA recipients, β is the discount factor, V E(1,0) is the value of the first period of employment, and V A is the value of being on Income Assistance. The only reason IA recipients are not employed is because an employment opportunity is not available and the only way an IA recipient can increase the likelihood of finding a job is through increased search effort. As we will see below the match rate m(0) is determined in part by search effort p(0).12

2.2 Unemployed individuals

Unemployed agents receive exogenous unemployment benefits (b u ) and pay search costs c u [p(i)z]. We make the simplifying assumption that unemployment benefits are independent of the individual’s pre-separation earnings. With probability m(i), individuals contact a firm with a vacancy and transit to employment in the next month. If individuals remain unemployed in the next month, it is assumed they can continue to collect unemployment benefits until benefits are exhausted. Following the last month of benefit eligibility, individuals can either transit to employment, if a job opportunity is available, or transit to IA. The value function for unemployed individuals with i months of benefits remaining is

$$V^{U}(i) =\left\{ \begin{array}{ll} \max_{p(i)} \left\{b_{u}-c_{u}[p(i)^{z}]\right. \\ \left.\qquad\qquad + \beta \left[m(i) V^{E}(1,i-1)+(1-m(i))V^{U}(i-1) \right]\right\} & 1 < i \leq \bar{u}, \\ \max_{p(i)} \left\{b_{u}-c_{u}[p(i)^{z}] \right.\\ \left.\qquad\qquad + \beta \left[m(1) V^{E}(1,0) +(1-m(1))V^{A} \right]\right\} & i=1. \end{array} \right. $$

2.3 Workers

The value of employment for a worker depends on her job tenure t and unemployment eligibility status i, where \(i \in \{0,1,\ldots,\underline {u},\ldots,\bar {u}\}\). As noted above, the number of months an individual with no benefits must work to qualify for unemployment is I. For every period an individual works beyond the qualifying period I, months of eligibility i increases by 1. The maximum number of benefit months an individual can accumulate is denoted \(\bar {u}\). If the individual were not working she would therefore be unemployed with i periods of benefits remaining. With probability δ, jobs are exogenously destroyed in the subsequent month, in which case workers transit to Income Assistance if they have not yet qualified for unemployment benefits (i=0) and transit to unemployment otherwise. With probability (1−δ) workers remain employed in the next month.

It is assumed that individuals who return to work before their unemployment benefits expire retain their remaining unemployment benefit eligibility. Finally, workers experience on-the-job wage growth for a maximum of T months, after which the wage remains constant, where it is assumed \(T>\bar {u}\). The value function for a worker with outside option i and with job tenure t is:

$$V^{E}(t,i)=\left\{ \begin{array}{ll} w(t,0)+\beta \left[(1-\delta)V^{E}(t+1,0)+\delta V^{A}\right] &~\text{if}~ t<I ~\text{and}~ i=0, \\ w(t,0)+\beta \left[(1-\delta)V^{E}\left(t+1,\underline{u}\right)+\delta V^{U}\left(\underline{u}\right)\right] & ~\text{if}~ t=I ~\text{and}~ i=0,\\ w(t,i)+\beta \left[(1-\delta)V^{E}(t+1,i)+\delta V^{U}(i)\right] & ~\text{if}~ 0 < i < \bar{u} \\ & ~\text{and}~ t < I, \\ w(t,i)+\beta \left[(1-\delta)V^{E}(t+1,i+1)+\delta V^{U}(i+1)\right] & ~\text{if}~ 0 < i < \bar{u} \\ & \text{and } I \leq t < T, \\ w(t,\bar{u})+\beta \left[(1-\delta)V^{E}(t+1,\bar{u})+\delta V^{U}(\bar{u})\right] & \,\text{if}\, i=\bar{u}\\ & \text{and}\, I \leq t < T, \\ w(T,\bar{u})+\beta \left[(1-\delta)V^{E}(T,\bar{u})+\delta V^{U}(\bar{u})\right] & \text{if}\, t \geq T, \end{array}\right. $$

where w(t,i) is the wage for a person with tenure t who has unemployment eligibility i.

2.4 Search technology

We assume there is no on-the-job search in the economy. The probability that a jobless individual receives a job offer depends on the probability the worker contacts a firm and the probability a firm has a vacancy. It is assumed that every firm employs at most one worker.

2.4.1 2.4.1 Workers

The probability a firm has a vacancy is simply the total number of vacancies divided by the total number of firms \(\frac {V}{F}\). Applications for jobs arrive according to a Poisson process, where λ is the average number of applications filed by workers at each firm. It is further assumed that firms randomly draw workers from the applicant pool if there is more than one applicant.13 The probability a worker is offered a job is:

$$\frac{1-e^{-\lambda}}{\lambda}. $$

The conditional re-employment probabilities for unemployed workers and workers on Income Assistance can then be expressed as the product of the above components, multiplied by the worker’s search effort

$$m(i)=\frac{p(i)V}{\lambda F}\left(1-e^{-\lambda}\right), $$

where

$$\lambda=\frac{1}{F}\left(\sum_{i=1}^{\bar{u}}p(i)U(i)+p(0)A \right), $$

U(i) is the number of unemployed workers with i months of unemployment insurance benefits remaining, A is the number of workers on IA, and p(0) and p(i) denote the contact probabilities for IA recipients and unemployed individuals with i periods of UI receipt remaining, respectively.14 The contact probabilities are choice variables for the workers within the model and can be interpreted as search effort. Workers determine the optimal level of search effort by equating the marginal benefit from an increase in search effort with its marginal cost.15 The optimal level of search effort, for each labor market state and program eligibility combination, is the solution to the following:

$$\begin{array}{@{}rcl@{}} p(0)&=&\left(\frac{\beta m(0)}{c_{a}z}\left[V^{E}(1,0)-V^{A}\right]\right)^{ \frac{1}{z}}, \notag \\ p(1)&=&\left(\frac{\beta m(1)}{c_{u}z}\left[V^{E}(1,0)-V^{A} \right]\right)^{\frac{1}{z}}, \text{} i=1, \\ p(i)&=&\left(\frac{\beta m(i)}{c_{u}z}\left[V^{E}(1,i-1)-V^{U}(i-1) \right]\right)^{\frac{1}{z}}, \text{}1 < i \leq \bar{u}. \notag \end{array} $$

The model presented above is a well-known model of the labor market. It contains features common to many unemployment and welfare programs and is a model that is straightforward to extend to study many policy reforms. In the next section, we evaluate the performance of the above aspects of the model using experimental data from the SSP.

3 Testing the model with experimental data

In this section, we describe a way to use social experiments as a test of the predictive power of the model. The social experiment we consider here is the Canadian Self-Sufficiency Project. The Self-Sufficiency Project provides an ideal application for this paper, as the policy implemented in the experiment is relatively straightforward to introduce in the model and no additional parameters need to be calibrated. We start by providing some details on the SSP and then outline our approach for conducting a partial equilibrium policy evaluation of the Self-Sufficiency Project.16

The Canadian Self-Sufficiency (SSP) experiment focused on long-term IA recipients.17 The universe for the experiment was long-term single parent IA recipients ages 19 and older in selected areas in British Columbia and New Brunswick from November 1992 to March 1995. This universe was sampled at random.18 Of those selected, 6,028 recipients volunteered to participate in the experiment and were subsequently placed in program and control groups by random assignment.19 Individuals assigned to the program group were informed that they were to receive an earnings supplement if they found a full-time (30 hours per week) job within one year and left Income Assistance. The supplement received by members of the program group depends on their labor market earnings.20 In particular, the supplement payment equals one-half of the difference between the earnings of the recipient and a benchmark earnings level, set at $37,000 in British Columbia and at $30,000 in New Brunswick for those earning less than the benchmark earnings level. Once individuals start receiving the supplement, they continue to do so for up to three years, as long as they remain employed full-time. Individuals in the program group who were not able to secure full-time employment within the twelve months following random assignment were not eligible to receive the supplement. Individuals in the control group were never eligible for the supplement.

The data contain information on 5,685 recipients in the main study: 2,827 control group members and 2,858 program group members.21 From the full sample, 4,371 single mothers provided information for the 18, 36, and 54 month follow-up surveys. We eliminate a further 1,025 observations for those individuals already employed at random assignment. The remaining sample contains 3,346 respondents, of whom 1,671 are members of the control group and 1,675 are members of the program group.22 This final sample is the one we use for our analysis.

The process we undertake to evaluate the model (and our calibration of it) involves the following three stages:

  1. 1.

    Calibrate the model to the populations targeted by the SSP social experiment and to the control group in the social experiment in each province. This represents the model control groups.

  2. 2.

    Introduce the Self-Sufficiency Project in the model as an experiment. Simulate the behavioral effects of the program in partial equilibrium for each province. This represents our model program groups.

  3. 3.

    Compare the levels and impacts predicted by the model to those observed in the data. This exercise provides evidence on how well our model and simulated experiment are able to replicate the impacts generated by the actual experiment. It is important to emphasize that the partial equilibrium version of the model is the appropriate comparison to the experiment because the experiment only affected a small subset of the economy and as such is not expected to have equilibrium impacts, as compared to a change in policy affecting all IA recipients.

Each step will be discussed in detail below.

3.1 Calibration of the model control group

In this section, we calibrate the model presented above to data on single mothers without completed post-secondary education23 and to data on the control group from the experiment. The model is calibrated separately for British Columbia and New Brunswick, the two provinces in which the SSP experiment was implemented. There are two main reasons that we chose calibration as opposed to estimation. The first is comparability. Calibration is a standard way to conduct a quantitative analysis using a DMP-style search and matching model. Since this literature has a great interest in the general equilibrium implications of policy interventions, we chose calibration in this case so that we could speak directly to this literature. Second, some of the key parameters that were calibrated would not have been identified non-parametrically given the data available in the SSP context.24 More broadly, for the most part we are using the same information to calibrate the model that we would have used had we estimated the model. So the parameter values from an estimation exercise would likely not differ substantially from those reported here.

The parameters for the partial equilibrium version of the model include monthly Income Assistance and unemployment benefits (b a and b u , respectively), the wage profile, the size of the labor force (L), the vacancy rate (V/F), the job separation rate (δ), the discount factor (β), and the search friction parameters (c a , c u , z). The values used for these parameters are all reported in Table 1. Monthly Income Assistance benefits (b a ) for single mothers are based on the average IA benefits for a single parent with one child during the 1990s, as reported in the National Council of Welfare Reports (2002). Over this period the average monthly Income Assistance benefit in British Columbia was $927 and in New Brunswick was $737. Unemployment insurance benefits (b u ) are set at 55 percent of average earnings. In both provinces, the earnings sample is limited to single mothers without completed post-secondary education, as we are attempting to isolate that segment of the labor market most similar to individuals receiving Income Assistance.25 The earnings data are based on the usual hourly wage, as reported in the monthly Labour Force Survey (1997-2000), assuming a 37.5 hour work week.26 , 27 We use data from the Labour Force Survey, as opposed to experimental data, as there is not enough wage data (in particular at jobs of long durations) available in the SSP to estimate the full wage distribution by tenure. We estimate a regression with the wage as the dependent variable and a cubic in tenure as the independent variables. We generate predicted values from the regression, for tenures of 1 to 48 months, and multiply the predicted values by 37.5 to obtain a full-time earnings value for each level of tenure. Full-time earnings for those with tenure greater than 48 months are set equal to full-time earnings for workers with tenure equal to 48 months. Earnings, IA benefits and unemployment benefits are all converted to 1992 dollars using the all-goods CPI.28 The resulting monthly unemployment benefits level is $952 in British Columbia and $695 in New Brunswick.

Table 1 Moments and parameters for single mothers without completed postsecondary education

The model is homogeneous of degree zero in L and F; we can therefore normalize the size of the labor force to 100 without loss of generality. The number of firms in the economy will be estimated in the baseline model and is identified using the observed vacancy rate in the economy. The following relation determines V endogenously as a function of F and E:

$$F=E+V. $$

In order to estimate F, we use the additional relationship between F and V given by the vacancy rate (v)

$$\frac{V}{F} = v. $$

The vacancy rate of 3.20 % is taken from Galarneau et al. (2001) and is based on the average for the retail trade and consumer services and labor-intensive tertiary manufacturing sectors, both of which have average incomes similar to our sample. Therefore, using the above equations, and for a given value of E,

$$F=\frac{E}{(1-0.032)}. $$

The job separation rate in the model (δ) is constant and can be directly estimated by the average job tenure for single mothers with no completed post-secondary education in the monthly Labour Force Survey (1990-2000). Job tenure is only reported for individuals currently employed in the data: we do not have direct information on separations. However, average job tenure is observed and in the model is equal to

$$\frac{\sum_{t=1}^{\infty}t E(t)}{E} = \frac{E(1) \sum_{t=1}^{\infty}t (1-\delta)^{t-1}}{E(1) \sum_{t=1}^{\infty}(1-\delta)^{t-1}}= \frac{1}{\delta}. $$

Average job tenure in the Labour Force Survey (LFS 1990–2000) implies a separation rate of 0.0214 in British Columbia and of 0.0121 in New Brunswick.29

We use parameter estimates for our search cost function from Christensen et al. (2005), whose estimates of the elasticity of search costs imply z=1.8457, and we set the monthly discount factor β equal to 0.9835, corresponding to an annual discount factor of 0.82 as in Davidson and Woodbury (1993). We assess the sensitivity of our results to these parameters in Section 4.4 below, as these parameters cannot be directly identified from the Canadian data. The costs of search are allowed to differ depending on whether individuals are receiving unemployment insurance or Income Assistance to capture the notion that searching may be less costly while unemployed. For example, unemployed individuals may have access to better search technologies through unemployment offices than do IA recipients, which would be consistent with c a >c u .

To identify the search friction parameter c a , we use the Income Assistance-to-work transition rate, m(0), directly observed from the SSP control group. Our model assumes that neither individual search effort nor the probability of a match vary with tenure on IA; put differently, it assumes a constant exit hazard from IA to employment. We show later on that this assumption matches the control group data well. The data on the control group do not provide information on the transition from unemployment to work necessary to identify the search friction for this group (c u ). Instead we use the search friction implied by the unemployment-to-work transition rate of the low skilled labor force.30

Next we must specify the length of time a worker is eligible for unemployment benefits. The length of the unemployment eligibility period depends on the unemployment rate in each province and on the worker’s previous job tenure. We set the eligibility periods in the model according to the eligibility rules during the 1990s. This implies that a worker is entitled to 5 (7) months of benefits after working 4 (3) months and 10 (12) months of benefits after working 9 (8) months or more in British Columbia (New Brunswick).31

3.2 Constructing the model treatment group

The following additions are made to the model to incorporate the Self-Sufficiency Project.32 First, individuals on IA face several time constraints. IA recipients become eligible for SSP after they have been on Income Assistance a minimum of 12 months. Once eligible for SSP, individuals have 12 months to find employment in order to receive supplement payments. If an individual secures a job before the eligibility period ends, she can receive the supplement while employed for a maximum of 36 calendar months. Consistent with the SSP treatment implemented in the experiment, individuals have one eligibility period for the treatment. Once the eligibility period for the supplement payments expires, individuals return to the regular IA system. Second, eligible individuals who find work receive supplement payments that are a function of their earnings upon obtaining employment. As in the baseline version of the model, earnings increase with job tenure. One goal of the SSP is to provide workers with enough time to experience sufficient earnings growth so that employment remains an attractive alternative once the earnings supplement expires. On-the-job earnings growth, which results from increases in the surplus created in worker-firm matches in our model, captures this particular feature of the program. In the following section, we assess the ability of the calibrated model to match the main features of the experimental data.

4 Comparing the moments of the model experiment with the moments of the social experiment

We now compare the predicted partial equilibrium effects of SSP to those found in the SSP experiment. This comparison represents the specification test of our model, in the same spirit as the comparisons of experimental and non-experimental partial equilibrium estimates in LaLonde (1986) and other, similar papers in the treatment effects literature. Throughout our empirical work, we treat the model predictions as constants when performing statistical tests comparing the model predictions to the SSP data. This serves to strengthen the inferences we draw from the test results by making the tests more conservative. It is important to emphasize that we do not use any information on the SSP program group in the calibration of our model; we are interested in determining whether the model can predict the behavior of the program group without exploiting the variation introduced by the experiment. Later, in Section 4.5, we show that using the extra variation introduced by the experiment affects the predictive performance of the model in New Brunswick but not British Columbia, further illustrating the usefulness of our model test.

In order to mimic the experimental design of the SSP, we solve the partial equilibrium model, using a fixed earnings profile, to obtain the conditional re-employment probabilities. After calibrating and simulating the model, we select those individuals who received Income Assistance benefits for 12 months. The re-employment probabilities from this simulated sample represent the simulated control group. We then use the calibrated parameters and solve for the re-employment probabilities for an individual on IA who is offered the SSP supplement. Again, this is done in partial equilibrium, implying any change in behavior will not have an impact on any other individuals, or on the earnings distribution.33 The re-employment probabilities in this instance represent the program group in our simulation.

4.1 Search intensity

As a test of the importance of endogenous search intensity, we compare the Income Assistance survival probabilities, at six month intervals, for the control and program groups in the social experiment and in the model simulation graphically in Fig. 1 and with formal tests in Table 2. If search intensity were fixed, then the transition rate from IA to work should remain unchanged once the SSP is introduced in the partial equilibrium version of the model (i.e., holding the wage distribution, hours, job destruction and vacancies constant). In our (relatively) simple model, in which we omit a reservation wage channel, endogenous search intensity is the only channel through which the transition rate can differ in this case; as such, we have a direct test of the assumption of endogenous versus fixed search intensity. We construct survival probabilities from the simulated model using the IA to employment transition rates for IA recipients in the simulated program and control groups. We start by discussing the results for British Columbia. The basic pattern matches the experimental data very well. Both the model and the experiment indicate that the SSP top-up reduces the Income Assistance survival rate substantially. During the 12 months of program eligibility, individuals increase their search effort and as a result transit to employment at a faster rate. Once the 12 month eligibility period is over, behavior reverts back to what it was in the absence of the program, and the transition rate to employment, conditional on tenure, returns to exactly what it was in the absence of the treatment. It is very encouraging that the model correctly predicts not only the basic pattern, but also matches the proportion of the program group remaining on Income Assistance 54 months after random assignment. For each month, we test the hypotheses that the model matches the experiment for the control group and for the program group, and the hypothesis that the simulated impact of the experiment matches the actual impact. The p-values for the joint hypothesis tests provide support for the model in this instance as the model is not significantly, or substantively, different from the experimental data at conventional significance levels.

Fig. 1
figure 1

Actual and Simulated Impact of SSP. The solid line is the data; the shaded area is a 95 % point-wise confidence interval; the dashed line is the simulation

Table 2 Income assistance survival probabilities

As an additional test of how well the model predicts behavior, we reproduce a second experiment designed to estimate the delayed exit from Income Assistance that may result from the 12 month qualification period. The delayed-exit experiment was a separate experiment conducted on a sample of 3,315 single parents in their first month of Income Assistance receipt in the metropolitan area of Vancouver, British Columbia. This sample was randomly assigned to program and control groups, where the program group was told that they would become eligible for the SSP program if they remained on Income Assistance for 12 months. The difference between the fraction remaining on Income Assistance in the program and control groups 12 months after random assignment is estimated by Ford et al. (2003) to be 3.9 percentage points with a standard error of 1.4. We conduct the same experiment in our model in partial equilibrium. The model predicts a delayed-exit effect of 4.3 percentage points in British Columbia, which is within one-third of one standard error of the effect estimated by Ford et al. (2003). The model is thus able to predict the magnitude of the experimental delayed-exit effect quite well. Comparing the model predictions with the experimental impacts, we can see that the model correctly predicts both the degree of delayed exit associated with the expectation of receiving the SSP benefit in the future (the entry effect) as well as the increased transition rate into employment that becoming eligible for the SSP program induces.

We come to a different conclusion upon examining the results for New Brunswick. As expected, the model provides a close fit to the control group. However, the simulated impact of the program on income assistance survival probabilities is significantly greater than that in the data. This is a substantial difference; the simulated impacts for months 12, 24, 36 and 48 are 2.76, 2.6, 2.52 and 2.8 times as large as the actual impacts. Our current parameterization of search intensity is not able to accurately predict the behavioral response to the introduction of the SSP in New Brunswick. We provide a more detailed explanation for this finding below.

4.2 Job destruction

Job destruction in the standard model is governed by a constant, exogenous job destruction rate.34 To evaluate this feature of the model, we compare the employment survival rates for the program and control groups in the data for those individuals who are employed 12 months after random assignment. Table 3 presents Kaplan-Meier survival rates for the program and control groups in the data. The first issue we consider is whether an exogenous job destruction rate is consistent with the experimental data. Two observations are of interest. First, formal statistical tests indicate that the survival rates are not significantly different for the program and control groups in the data.35 As in the model, the experimental data do not suggest the SSP has an impact on the job destruction rate in either province. Further, the data do not suggest that the job destruction rate increases after workers stop receiving supplement payments. Both features of the data are consistent with an exogenous job destruction rate in this context.

Table 3 Employment survival probabilities, conditional on being employed in month 12 following random assignment

However, the data in Table 3 on both the program and control groups do not support the assumption of a constant job destruction rate. For both the program and control groups, the employment exit hazard rate is decreasing with the duration of employment in both provinces. There are several possible explanations behind this pattern. One possibility is that there is heterogeneity in job destruction rates across jobs; over time, the sample of remaining matches contains a disproportionate number of jobs with low destruction rates. Another possibility is that workers and firms learn about the true match value over time, so that low quality matches separate at a faster rate, as in Jovanovic (1979) and Jovanovic (1984).

4.3 Earnings

One question of interest is whether earnings adjust in response to the introduction of an earnings supplement. In the partial equilibrium version of the model we consider, it is assumed earnings do not adjust to the policy change in the experiment because the number of individuals who receive the supplement is small. This assumption appears to be supported by the data, as there are no statistically significant differences between the (conditional on employment) earnings of the program and control groups in the experimental data in British Columbia and a significant difference only in month 6 for New Brunswick, as illustrated in Table 4.

Table 4 Average earnings, wages and hours

Taking a closer look at the data, however, provides a slightly different picture. In British Columbia in particular, hourly wages for the program group are substantially (and significantly) lower than those for the control group 12 months after random assignment, almost $2 per hour lower in this month. There are few differences in hourly wages between the program and control groups in New Brunswick, as individuals in each group enter employment at wages that are closer to the minimum wage. When differences do exist, they are less than $0.75 per hour. Program group members work substantially (and significantly) more hours than members of the control groups in both provinces, reflecting the hours requirement for collecting supplement payments. In month 12, program group members in British Columbia work an average of 17 more hours per month, while in New Brunswick they work an extra 10 hours per month. By the time supplement payments expire, there are no significant differences in wages or hours across the program and control groups. In addition, the joint tests suggest there are no significant differences in hours for British Columbia and wages in New Brunswick.36

The data therefore provide limited evidence to suggest individuals in the program group respond to the introduction of the earnings supplement by choosing a combination of lower wages and higher hours that ends up providing the same earnings, conditional on employment, as the higher wage/lower hours bundle that is chosen by the control group. Our simple model does not allow for both wages and hours to be determined in the bargaining process, thus we cannot use this model to explain these features of the data. The evidence from the experimental data suggests that the intensive margin may be an interesting margin to consider, especially if reforms of interest depend on hours requirements.

4.4 Sensitivity analysis

We next consider the sensitivity of our results to changes in the two parameters that are not calibrated to match data on the control group or Canadian labor force statistics: β and z. Both parameters were taken from the search literature, the former from a US study (Davidson and Woodbury, 1993) and the latter from estimates obtained using Danish data (Christensen et al., 2005). It is these two parameters that we therefore have the least confidence in. Table 5 presents evidence on the extent to which our results are sensitive to the choice of the discount factor (β) and the elasticity of search costs with respect to search effort (z). It is worth emphasizing that the search friction parameter c a is re-calibrated in the baseline model for each combination of β and z. Lower values for β serve to reduce the incentives of individuals on unemployment and those on IA to search as the value of employment falls. Lower values of c a are then required to match the transitions into employment from Income Assistance. A similar argument holds for z. If search costs become more elastic (a higher value for z), then lower search costs are necessary to match the transitions into employment. The re-calibrated values for c a are presented in the top row of each panel in Table 5.

Table 5 Sensitivity to the choice of β and z: IA survival probabilities

The remaining rows in Table 5 contain p-values for the following three hypothesis tests. First, we test whether the simulated survival rate for the control group in the model is equal to the survival rate for the control group in the data. Note that the p-values are the same for each parameterization. We recalibrate search costs when β and z change to match the control group data and, as a result, can achieve the same fit in each case. Second, we conduct the same set of hypothesis tests for the program group. Finally, we test whether the simulated impact is equal to the experimental impact. Each test is conducted at 12, 36, and 53 months. We also test the hypothesis that the total number of exits up to month 53 is the same in the data and in the model.

In the case of British Columbia, the results indicate that the simulated control group does not match the experimental control group at the 12 month mark. Thus, the in-sample tests are able to rule out some aspects of all the parameterizations considered here. However, assessing whether the model fits the data based only on in-sample fit can be quite misleading. Two examples of this point are readily observed. First, in several cases the model fits the control group but cannot replicate the impact of the program. The set of results for the case where z=1.500 and β=0.820 for British Columbia is illustrative of this point. The model is able to match the control group at 36 and 53 months; however, the model is not able to match the program group nor the experimental impacts in the data. In the absence of experimental data, we would be ignorant of the failure of the model to generate accurate predictions in this instance. This result highlights the fact that in-sample fit would not reject some parameterizations that our test would reject.

Second, in some cases, the simulated control group does not consistently match the data despite the fact that the model accurately predicts the impact of the SSP, as in the final column for British Columbia. It appears we are missing some features of the data that do not vary across the program and control groups and thus cancel out when computing the program impact. In this case, we may reject the model because of its inability to match the control group when in fact it is able to produce a good estimate of the program impact.

For New Brunswick, the sensitivity analysis indicates that the choice of a high elasticity of search costs with respect to effort provides a much better fit to the program group, with no change in fit to the control group. This result suggests that a combination of low search costs and a high value of z is necessary to fit the data for New Brunswick. It is again clear that the experimental data provides us with useful information on the performance of the model under various parameterizations.

4.5 Using the extra variation introduced by the experiment

Social experiments provide a source of exogenous variation that can be used to identify parameters of interest. In this paper, we use other sources of data for identification and instead use the exogenous variation introduced by the experiment to test the performance of our model. This alternative use of the experimental data is a convincing way to confirm the model’s ability to capture the responses of agents to new policy changes. In this section, we determine how well the model would fit the data if we were instead to use this additional variation in the calibration process. To this end, we conduct a grid search over z to determine what parameter values best fit the experimental impact in the data.37 A comparison between the performance of our benchmark model and that of the ‘optimal’ calibration is presented in Table 6. For British Columbia, we find the ‘optimal’ parameter values are very close to the benchmark values and that we only make a small improvement in model fit by taking advantage of the experimental variation. In the case of New Brunswick, the optimal parameter values are quite different from the benchmark values. The results from the former province suggest that ignoring the experimental variation in the calibration process does not affect the predictive performance of the model; the results from the latter suggest the opposite. The fact that the conclusions across provinces are mixed, despite the fact that identical experiments were implemented in both provinces, is further evidence of the usefulness of our model test.

Table 6 Comparison of benchmark and optimal calibration: IA survival probabilities

5 Conclusion

In this paper, we use social experiments as an innovative and rigorous test of a model’s ability to replicate observed behavior. In particular, we consider the ability of a standard search model, fit to an experimental control group, to match the program group as a formal test of the model. Our results provide strong support for the model in this instance along some dimensions but not others. First, we use evidence on the impacts of the SSP on the IA-to-work transition rates during the 53 months following random assignment as a test of the specification for search intensity in the model. We find that the model program and control groups are not significantly, nor substantively, different from those in the experimental data for British Columbia but are different for New Brunswick under our benchmark calibration. The model is also able to match the delayed-exit effects of a second randomized experiment that offered SSP to new Income Assistance recipients. Secondly, we find that the employment survival rates in the experimental data are consistent with an exogenous, but not a constant, job destruction rate. Our analysis also points to several extensions that may help in explaining other features of the data, including the joint determination of hours and wages.

It is important to emphasize two points. First, we could not draw this conclusion without the availability of experimental data. Our analysis highlights a second use of experimental data that has not been widely exploited in the literature. Second, the experimental data provide strong support for the search framework in this instance along certain dimensions, most notably the IA-to-work transition in British Columbia, the main margin over which the SSP is likely to influence behavior. Our model test indicates that the model captures the fundamental dynamics introduced by the SSP supplement. The data also support the use of the model in the case of New Brunswick, but under a much different set of parameter values. Considering the widespread use of this model, the results provide support for the use of this framework to study policy changes that impact the labor market. It is very important, however, that future work considers the robustness of this result by exploiting other sources of variation.

It is also worth emphasizing that our model test involves a direct test of a partial equilibrium version of the model. The reason for this is simple: the social experiment is conducted on a small sample of individuals and therefore does not have spillover effects on the rest of the economy. As such, we do not have a direct test of the equilibrium implications of the potential policy change. That said, we take the fact that the partial equilibrium version of the model passes our more rigorous test as very convincing evidence in favor of using the model for British Columbia (but not for New Brunswick), as the model can replicate many of the outcomes produced by a social experiment without the use of the variation introduced by the social experiment. This finding increases our confidence in equilibrium policy evaluations and the evaluation of potential policies that can be conducted within the model but were not conducted within the experiment. In a companion paper (Lise et al. 2004) we use the model presented here to evaluate the potential spillover effects that may arise should the SSP be implemented as a policy on a wide scale for British Columbia. Lise et al. (2004) find that several important feedback effects, including displacement and changes in the equilibrium wage distribution, reverse the cost-benefit conclusions implied by the partial equilibrium experimental evaluation. Taken together, both papers illustrate that combining social experiments and models of the labor market and government assistance programs represents a powerful tool for policy evaluation.

6 Endnotes

1 For example, see the evaluation of the Reemployment Bonus by Davidson and Woodbury (1993) and many calibrated studies based on the DMP model such as Cole and Rogerson (1999) and den Haan et al. (2000). Similarly, Plesca (2010) evaluates the US Employment Service (i.e., labor exchange), Albrecht et al. (2009) estimate the equilibrium effects of the large-scale Swedish “Knowledge Lift” program, and Cahuc and Le Barbanchon (2010) evaluate counseling for the unemployed, all in the context of DMP-style equilibrium models. Yashiv (2006) considers the within-sample fit of the DMP model. Others, such as Ljungqvist and Sargent (1998) and Alvarez and Veracierto (2000), have considered a variety of labor market policies using a modification of the McCall (1970) search framework. Hopenhayn and Rogerson (1993) consider policy evaluation in a calibrated equilibrium model using firm-level data.

2 A very useful discussion of this and related issues can be found in Hansen and Heckman (1996).

3 See Kydland and Prescott (1996) for a detailed discussion.

4 It is also possible to estimate a structural model on a subset of the data, for example a subset of states, and use the remainder of the data to conduct a validation exercise. See Keane and Wolpin (2007).

5 Crépon et al. (2013) provide experimental estimates of the equilibrium effects of active labor market programs in France using a unique design that randomizes the fraction treated at the level of the labor market and individual treatment assignment within labor markets.

6 The ability of the canonical DMP model to fit to the data has recently attracted much attention. A large literature has now developed, starting from Shimer (2005) and summarized nicely in Mortensen and Nagypal (2007), on what model features are necessary for the DMP model to replicate aggregate fluctuations in productivity, unemployment and vacancies. The simplest version of the DMP model with Nash bargaining and homogeneous workers and firms does not produce much volatility in unemployment and vacancies in response to fluctuations in labor productivity. This can be overcome with, for example, alternative assumptions on wage determination (Hall 2005; Gertler and Trigari 2009) or worker and firm heterogeneity (Lise and Robin 2014). Recently Hornstein et al. (2011) illustrate quite clearly that without on-the-job search, this class of models produces very little in the way of wage dispersion.

7 In parallel work, Todd and Wolpin (2006) use data from the experimental evaluation of Mexico’s PROGRESA program to test a structural, dynamic model of fertility and child schooling using a similar approach. Bajari and Hortaçsu (2005) test a structural auction model using data from a laboratory experiment. Earlier work by Wise (1985) uses experimental data on housing subsidies to test a model of housing demand. More recently, Attanasio et al. (2012) use experimental data in the estimation of their equilibrium model of PROGRESA, and Gautier et al. (2012) combine experimental data with data on non-participants from labor markets affected and not affected by the experiment to estimate the general equilibrium effects of a job search treatment using a differences-in-differences design.

8 A large literature considers various aspects of the Self-Sufficiency Project, including Bitler et al. (2008), Card and Hyslop (2005), Connolly and Gottschalk (2009), Ferrall (2012), Foley and Schwartz (2003), Kamionka and Lacroix (2008), Michalopoulos et al. (2005), Riddell and Riddell (2014), and Zabel et al. (2013). Similar, but less generous, income supplement programs have been studied in the United States; see Auspos et al. (2000) on the Minnesota Family Investment Program and Bos et al. (1999) on the Wisconsin New Hope program. Bloom and Michalopoulos (2001) provide an overview of the experimental literature and compare these programs to other approaches.

9 We do not consider distributional effects of the policy change on earnings, as the simple model is not well suited to doing so.

10 Throughout this paper we use the term unemployed to mean collecting unemployment benefits. In the model, all jobless individuals are actively seeking employment; they are distinguished by whether they are receiving unemployment benefits or Income Assistance benefits.

11 The firm’s problem and the equilibrium conditions are outlined in detail in the Appendix. Section A.2 of the appendix describes how wages are set via Nash bargaining between the firm and the worker in the context of the minimum wage in the equilibrium version of the model. We also implicitly assume that the value of employment always exceeds the value of being on IA or unemployed so that a match forms whenever a worker successfully contacts a firm.

12 It is worth noting that we assume all Income Assistance recipients enter employment with tenure zero. This assumption could be relaxed by assuming workers retain their experience when they enter Income Assistance and allowing experience to depreciate over time. One implication of this extension would be that the hazard of leaving IA would decline as the length of the IA spell increased. The drawback of this extension is that it involves a large increase in the size of the state space.

13 Alternatively, we can consider the length of a period tending to zero and work in continuous time, where there is zero probability of more than one application arriving simultaneously.

14 Note that our matching function, as in Davidson and Woodbury (1993), exhibits increasing rather than constant returns to scale. While this is a somewhat nonstandard assumption, it has no effect on our results since our tests are done in partial equilibrium. We could have equivalently assumed that the job contact probability is simply proportional to search effort.

15 In determining the marginal cost and benefit of search effort λ is held constant under the assumption that each worker believes her impact is small relative to total labor supply.

16 For comprehensive details on the Self-Sufficiency Project, see Michalopoulos et al. (2002).

17 In particular, individuals had to receive IA in the current month and in at least 11 out of 12 of the prior months to be included in the experiment.

18 The selected areas were the lower mainland in British Columbia and the lower third of New Brunswick.

19 As noted in Lin et al. (1998), about 90 percent of the target population agreed to be randomly assigned as part of the main (“recipient”) component of the SSP demonstration that constitutes our primary focus in this paper. This leaves only modest scope for biases in the experimental impact estimates when interpreted as applying to the entire target population rather than just the study population of those who agree to random assignment. Kamionka and Lacroix (2008) consider the smaller “early entry” experiment, which we briefly consider in Section 4.1. It attracted only about 80 percent of its target population into random assignment. Their analysis suggests that this leads to a substantively meaningful understatement of the effects of the SSP treatment. Sianesi (2014) provides further useful discussion of these issues, an application to a more recent experiment, as well as pointers to the broader literature.

20 No other sources of income affect the calculation of the earnings supplement.

21 Out of the full sample of 6,208 individuals participating in the baseline interview, 2,849 individuals were assigned to the control group, 2,880 were assigned to the program group, and the remaining 299 were assigned to a third experimental group, the SSP Plus group. Thus, we are missing information on 464 respondents. A total of 40 respondents did not meet the criteria for inclusion in the experiment, and 3 control group members withdrew. An additional 398 individuals could not be contacted or refused to be interviewed. Lin, Robins, Card, Harknett, and Lui-Gurr (1998) consider the possible nonresponse biases that may arise and conclude the biases are likely to be small.

22 The control group contains 815 recipients from New Brunswick and 856 from British Columbia, while the program group consists of samples of 813 and 862 respondents from New Brunswick and British Columbia, respectively.

23 “Without completed post-secondary education” refers to respondents reporting up to some post-secondary education but no post-secondary certificate or higher.

24 For example, the search intensity parameter (z) would only be identified off the non-linearity of search costs in preferences in the absence of matched worker-firm data. The same type of identification strategy would have to be applied in the estimation of the discount factor.

25 Approximately 14 % of the control group in the Self-Sufficiency Project has at least some post-secondary education: 11.3 % of the sample reports having some post-secondary education and 2.2 % of the sample reports having a degree or certificate from a university.

26 The standard full-time weekly hours in Canada is 37.5 hours.

27 The Canadian Labour Force Survey is the analogue of the U.S. Current Population Survey.

28 All figures are reported in Canadian dollars, where $1Cdn was approximately equal to $0.63US at the time of the experiment.

29 This measure of job tenure does not take quits into account. As a result, we may overestimate the job separation rate as individuals moving between jobs because of quits report holding jobs of shorter durations.

30 See Lise et al. (2004) for further details.

31 See Lin et al. (1998).

32 The model changes required to introduce the SSP are minor. Full details on the augmented model are presented in the Appendix.

33 For example, this means that λ does not change when we introduce the program in partial equilibrium. We also assume that the introduction of the SSP is not anticipated by individuals in the simulated program group.

34 See Mortensen and Pissarides (1994) for a model with endogenous job destruction.

35 See the notes at the bottom of Table 3 for full details.

36 As noted by Card and Hyslop (2005) and Zabel et al. (2013), the fact that wages can only be compared for those members of the program and control groups that work may be problematic. In the spirit of most relatively simple search and matching models in the DMP tradition, we ignore issues surrounding unobserved heterogeneity in this paper.

37 The grid is over z{1.50,3.50} with an interval size of 0.005. We do not search over both β and z, as the discount factor is not separately identified from search costs. For each value of z we consider, the search costs parameter c a is recalibrated to match the IA survival function for the control group. Our measure of best fit is the maximum p-value for the joint test (all months) of simulated versus actual impacts for IA survival probabilities.

38 We assume for simplicity that individuals entering employment from unemployment with remaining benefit months available start accumulating benefit months immediately.

39 If the individual’s earnings are above the ceiling, they do not receive a supplement. For simplicity, we abstract from this case within the model as few individuals had earnings above the supplement ceiling in the data. In the model no individuals have earnings above the supplement ceiling.

40 In the actual SSP experiment, once individuals qualified for the earnings supplement they could transit between employment and IA and collect the supplement payments in any month they were employed full-time during the 36 months after qualifying. We abstract from this in the model as it would add an unmanageable number of states. Instead, we allow those who lose their job prior to qualifying for unemployment benefits to transit back to the first period of SSP eligibility.

41 It is assumed that I<T end , which is consistent with the actual EI and IA programs.

42 Production takes place when there is a match between one firm and one worker; the number of firms can alternatively be interpreted as the number of jobs in the economy.

43 Alternatively, we can consider the length of a period tending to zero and work in continuous time, where there is zero probability of more than one application arriving simultaneously.

44 In determining the marginal cost and benefit of search effort λ is held constant under the assumption that each worker believes her impact is small relative to total labor supply.

45See Judd (1998) Ch.4, p. 114 and Ch.5, pp. 168–171.

7 Appendix A: The basic model

7.1 A.1 Firms

Production takes place when there is a match between one firm and one worker; the number of firms can alternatively be interpreted as the number of jobs in the economy. There is free entry in the economy. In every period, each firm has the option of filling a vacancy, if one exists, by hiring a worker or keeping the vacancy open. If matched with a worker, firms earn profits that depend on the surplus generated by the match and pay wages, determined in equilibrium, that depend on the worker’s outside options and the minimum wage. Profits depend on the worker’s tenure to allow match-specific capital to increase the productivity of the match over time. Denote the surplus generated by a worker-firm pair of tenure t by S(t). With probability δ the match separates and the firm is left with a vacancy in the following month. Denote the profits of a firm matched with a worker with outside option i, \(i \in \{0,1,\ldots,\underline {u},\ldots,\bar {u}\}\) and match tenure t as Π(t,i).

The expected discounted present value of profits for matches of job tenure t and workers with outside option i are

$${}\Pi^{E}(t,i)=S(t)-w(t,i) + \begin{cases} \beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,0) ] & ~\text{if}~ i=0 ~\text{and}~ t<I,\\ \beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\underline{u}) ] & ~\text{if}~ i=0 ~\text{and}~ t=I, \\ \beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,i+1) ] &~ \text{if}~ 0<i<\bar{u} ~\text{and}~ t<T, \\ \beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\bar{u}) ] & ~\text{if}~ i=\bar{u} ~\text{and} ~t<T, \\ \beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(T,\bar{u}) ] & ~\text{if}~ t \geq T, \end{cases} $$

where match tenure beyond T no longer increases profits.

If a firm has a vacancy, the value of the vacancy is determined by the probability of meeting an unmatched worker, by the profits the firm expects to make from the match, and by the costs of posting a vacancy (ξ)

$$\Pi^{V}= - \xi + \beta \left[\sum_{i=0}^{\bar{u}}q(i)\Pi^{E}(1,i) +\left(1-\sum_{i=0}^{\bar{u}}q(i)\right)\Pi^{V}\right], $$

where q(i) is the probability a firm matches with a worker with outside option i.

7.1.1 A.1.1 Matching probabilities

From the firm’s perspective, the probabilities of meeting potential workers from unemployment and IA equal the fraction of workers from unemployment and IA who transit to employment, divided by the total number of vacancies

$$q(i)=\frac{m(i)U(i)}{V} \qquad \text{and} \qquad q(0)=\frac{m(0)A }{V}, $$

respectively.

7.2 A.2 Equilibrium wage determination

After meeting in the labor market, a firm and a worker bargain over wages by making alternating wage offers until both sides find the offer acceptable. It is assumed that both parties have equal bargaining power, but have different threat points. The equilibrium of this game is the Nash cooperative bargaining solution and results in workers and firms splitting the surplus of a match evenly. The surplus of the match from the worker’s perspective is the difference between employment at the equilibrium wage and the worker’s outside option, which depends on their current labor market state and program eligibility. The surplus from the perspective of the firm is the difference between the profits the firm receives at the equilibrium wage and the value of leaving the vacancy open. It is further assumed that the bargaining process is constrained such that the wage can not fall below the minimum wage \(\underline {w}\). The equilibrium wage is \(\max \{w(t,i), \underline {w}\}\), where w(t,i) solves

$$V^{E}(t,i)-V^{i}=\Pi^{E}(t,i)-\Pi^{V}, $$

where V i{V A,V U(i)} is the value of the outside option i. In the following section, we define the steady state conditions for employment, unemployment, and IA.

7.3 A.3 Steady state conditions

Let E denote the steady state number of jobs occupied by workers and V the number of vacancies. By definition, the total number of jobs in the labor market is equal to the total number of occupied jobs and the total number of vacancies

$$F=E+V. $$

Denote the total number of individuals in the labor market by L. The total number of individuals can be decomposed into three groups. First the employed, who are distinguished both by their current job tenure and their current outside option

$$E=\sum^{T}_{t=1} \sum_{i=0}^{\bar{u}} E(t,i)+\bar{E}, $$

where \(\bar {E}\) is the group of workers no longer experiencing on-the-job wage growth. The second group, denoted A, consists of individuals on Income Assistance. The final group are unemployed individuals (U)

$$U=\sum^{\bar{u}}_{i=1}U(i), $$

where U(i) indicates the number of unemployed persons with i periods of benefits remaining.

The total number of individuals in the labor market can therefore be expressed as the sum of the above components

$$L=E+A+U. $$

Using the above definitions, we can describe the conditions governing the steady state, where the flows in and out of every employment state must be equal over time. The steady state conditions for each state and eligibility combination are discussed in turn below.

7.3.1 A.3.1 Employment

As above, m(0) and m(i) denote the probabilities that IA recipients and unemployment recipients with i periods of benefits remaining, respectively, match with a firm. The flow into the first period of employment includes those workers from IA and unemployment who receive job offers. They are indexed by their respective outside options as this will determine their progression of benefit entitlements.38 In subsequent periods, the inflow consists of workers who were employed in the previous period and who were not exogenously separated from their jobs

$$\begin{array}{@{}rcl@{}} E(1,0) &=& m(0)A + m(1)U(1) \\ E(1,i) &=& m(i+1)U(i+1), \quad 0 < i < \underline{u}\\ E(t,0) &=& (1-\delta)E(t-1,0), \quad 1 < t < I ~\text{and}~ i=0\\ E(t,i) &=& (1-\delta)E(t-1,i-1), \quad 1 < t < T ~\text{and}~ 0 < i \le \bar{u} \\ E(t,\bar{u}) &=& (1-\delta)E(t-1,\bar{u}), \quad 1 < t \leq T ~\text{and}~ i > \bar{u} \\ \delta \bar{E} &=& (1-\delta)E(T,\bar{u}). \end{array} $$

7.3.2 A.3.2 Income assistance

The flow into Income Assistance includes those employed workers who were exogenously separated from their jobs and ineligible for unemployment benefits and unemployed workers no longer eligible for unemployment benefits. The flow out of IA includes IA recipients who find employment. The steady state condition for IA is the following:

$$\delta \sum_{t=1}^{I-1} E(t,0)+(1-m(1))U(1)=m(0) A. $$

7.3.3 A.3.3 Unemployment

Employed workers who are separated from their jobs and who are eligible for the maximum months of unemployment benefits flow into the first period of unemployment, \(U(\bar {u})\). For U(i) where \(0 < i < {\bar {u}}\), the inflow consists of unemployed workers from the previous period who did not find jobs and workers separated from their jobs who qualify for less than the maximum number of benefit months. All workers flow out of the unemployment state when benefits run out due to the time limitations in the unemployment insurance program

$$\begin{array}{@{}rcl@{}} \delta \sum_{t}E(t,\bar{u}) &=& U(\bar{u}) \\ \delta \sum_{t}E(t,i-1) + (1-m(i+1))U(i+1)&=&U(i)~\text{if}~ 0<i<\bar{u}. \end{array} $$

8 B Model with self-sufficiency project

The welfare program and the earnings supplement are modeled as follows. First, in addition to individuals on unemployment insurance, individuals on welfare also face several time constraints. Second, individuals receive supplement payments that are a function of the wage upon obtaining employment.

Within the model, agents maximize expected lifetime income by choosing their labor market state and the intensity with which they search for work if not employed. It is assumed that agents have no information regarding the existence and structure of the SSP until month 12 of their IA spell. Workers bargain with firms over wages that depend on the tenure of the match and on the outside options of both parties. Through this channel, the model generates predictions regarding how starting wages vary depending on whether an individual is entering employment through EI or IA and on whether or not the individual is eligible for the supplement. One goal of the SSP is to provide workers with enough time to experience sufficient wage growth so they have an incentive to stay employed once the earnings supplement expires. On-the-job wage growth, through an increase in the surplus created in worker-firm matches, captures this particular feature of the program and is also incorporated in the model.

It is useful to expand the notation used in the base model in the following way. Let i index the jobless state an individual is currently in: \( i \in \{\bar {u},\ldots,\underline {u},.\,\ldots,1,0,-1,\ldots, -T_{\textit {in}},\ldots,-T_{\textit {out}}\}\). Here positive i indicates the individual is jobless with i unemployment benefit months remaining. We use 0≥i≥−T out to index the welfare recipients, where the key indices are i=0 in the first month on welfare, i=−T in in the first month of SSP eligibility, and i=−T out in the post-eligibility period.

8.1 B.1 Workers

The value of employment for a worker depends on her job tenure t and program eligibility status i, where \(i \in \{\bar {u},\ldots,\underline {u}.\ldots,1,0,-1,\ldots, -T_{\textit {in}},\ldots,-T_{\textit {out}}\}\). The number of months an individual with no benefits must work to qualify for unemployment is I. For every period an individual works after qualifying for benefits, i increases by 1. The maximum number of benefit months an individual can accumulate is denoted \(\bar {u}\). If the individual were not working she would therefore be unemployed with i periods of unemployment benefits remaining, \(i \in \{0,\ldots \bar {u}\}\). With probability δ, jobs are exogenously destroyed in the subsequent month, in which case workers transit to welfare if they have not yet qualified for unemployment benefits, i=0, and transit to unemployment otherwise. With probability (1−δ), workers remain employed in the next month.

It is assumed that individuals who return to work before their unemployment benefits expire retain their remaining unemployment benefit eligibility. Finally, workers experience on-the-job wage growth for a maximum of T months, after which the wage remains constant, where it is assumed \(T>\bar {u}\). The value function for a worker needs to account for job tenure t, outside option i, and earnings supplement receipt s{0,1}, where s=0 indicates the worker is not eligible to receive the supplement.

8.1.1 B.1.1 Workers that receive supplement payments

Employed workers who receive the supplement obtain labor market earnings and the supplement payment in the current month. The supplement received by the worker is a function of her wage. In particular, the worker receives one-half of the distance between her wage and an exogenous ceiling denoted \(\tilde {w}\); in other words, she receives the average of her market wage and the wage ceiling.39 Individuals remain eligible for the supplement for T end months once they leave Income Assistance and become employed. Therefore, workers continue to receive the supplement as long as the duration of the current employment spell is shorter than the allowed duration of the supplement payment period. With probability δ workers are exogenously separated from their jobs at which point they can transit to unemployment if eligible for EI or back to the first period of SSP eligibility (i=−T in ) if they do not yet qualify for unemployment benefits (t<I).40 Employed persons no longer receiving the supplement (\(t> T^{E}_{\textit {end}}\)) remain employed, or if they are exogenously separated from their job, transit to EI.41 The value of being employed and in the states described above can be expressed as

$${}V^{E}(t,i,1)\,=\,\left\{ \begin{array}{ll} \frac{1}{2}[w(t,-T_{in},1)+\tilde{w}] \\ +\beta \left[(1-\delta)V^{E}(t+1,-T_{in},1)+\delta V^{A}(-T_{in})\right] & ~\text{\!if}~ t<I, \\ \frac{1}{2}[w(t,-T_{in},1)+\tilde{w}]+\beta \left[(1-\delta)V^{E}(t+1,\underline{u},1)+\delta V^{U}(\underline{u})\right] & ~\text{\!if}~ t=I,\\ \frac{1}{2}[w(t,i,1)+\tilde{w}]+\beta \left[(1-\delta)V^{E}(t+1,i+1,1)+\delta V^{U}(i+1)\right] & ~\text{\!if}~ \underline{u} < i < \bar{u} \\ \frac{1}{2}[w(t,\bar{u},1)+\tilde{w}]+\beta \left[(1-\delta)V^{E}(t+1,\bar{u},1)+\delta V^{U}(\bar{u})\right] & ~\text{\!if}~ i=\bar{u} \\ \frac{1}{2}[w(t,\bar{u},1)+\tilde{w}]+\beta \left[(1-\delta)V^{E}(t+1,\bar{u},0)+\delta V^{U}(\bar{u})\right] & ~\text{\!if }~ t = T_{end}, \end{array}\right. $$

where T in is the number of months on welfare required to qualify for the SSP supplement and T end is the number of months an individual can receive the supplement once employed.

8.2 B.2 Welfare recipients that are eligible to receive the supplement

Welfare recipients receive welfare benefits (b a ) and pay convex (z>1) search costs c a [p(i)z] in the current month. The cost of search is modelled in a manner consistent with Davidson and Woodbury (1993), where z is the elasticity of search costs with respect to search effort, c a is a parameter capturing the disutility of search effort and p(i) is search effort in period i. The cost of search depends directly on the intensity with which workers search within the model. Let m(i) denote the probability a welfare recipient finds a job and transits to employment in the next month. With probability (1−m(i)) the welfare recipient remains on welfare. Welfare recipients eligible for the supplement receive welfare benefits in the current month. Using the same index i as we used for number of months of unemployment benefits, we define the first month on Income Assistance as i=0, and continue to count down. Individuals are eligible for the earning supplement in month i=−T in and become ineligible in month i=−T out . They remain eligible for the supplement as long as the duration of their welfare spell is less than the supplement eligibility period (−T in i>−T out ). For the duration of the time that they are eligible for the supplement, they receive a job offer with probability m(i), and if an offer is received, they have the option of transiting to employment in the next month or remaining on IA. If they do not receive a job offer, they remain on welfare and, if the eligibility period has not expired, remain eligible to receive the supplement should they secure employment in the following month. If eligibility for the supplement expires in the next month and workers do not receive a job offer, they remain on welfare but are not eligible for the supplement should they receive a job offer. Once eligibility expires, individuals will not be eligible for the earnings supplement for the remainder of their duration on IA. The value function for welfare recipients in the states described above is

$${}V^{A}(i)=\left\{ \begin{array}{ll} \max_{p(i)} \left\{b_{a}-c_{a}[p(i)^{z}]\right. \\ \left.\quad \quad +\beta \left[m(i)V^{E}(1,i-1,1)+(1-m(i))V^{A}(i-1)\right]\right\}, & -T_{in} \geq i > -T_{out} \\ \max_{p(i)} \left\{b_{a}-c_{a}[p(i)^{z}]\right. \\ \left.\quad \quad +\beta \left[ m(i)V^{E}(1,0,0)+(1-m(i))V^{A}(-T_{out})\right]\right\}, & i=-T_{out} \\ \end{array}\right. $$

8.3 B.3 Firms

In every period, the firm has the option of filling a vacancy, if one exists, by hiring a worker or keeping the vacancy open. If matched with a worker, firms earn profits that depend on the surplus generated by the match and pay wages, determined in equilibrium, that depend on the worker’s outside options, the minimum wage, and the worker’s SSP supplement status. Profits depend on the worker’s tenure to allow match-specific capital to increase the productivity of the match over time. Denote the surplus generated by a worker-firm pair of tenure t by S(t). With probability δ the match separates and the firm is left with a vacancy in the following month. Denote the profits of a firm matched with a worker with outside option i, \(i \in \{\bar {u},\ldots,\underline {u}.\ldots,1,0,-1,\ldots, -T_{\textit {in}},\ldots,-T_{\textit {out}}\}\), match tenure t, and SSP supplement eligibility s by Π(t,i,s).

The expected future profits for matches of job tenure t with workers with outside option i and SSP eligibility s are

$${}{\fontsize{8.8pt}{9.6pt}\selectfont{\begin{aligned} \Pi^{E}(t,i,s)=\left\{ \begin{array}{ll} S(t)-w(t,0,0) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,0,0) ] & ~~\text{if}~~ 0 \geq i > -T_{in}, t<I, \\ &~~\text{and}~~ s=0,\\ S(t)-w(t,0,0) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\underline{u},0)] & ~~\text{if}~~ i=0, t=I, ~~\text{and}~~ s\!=0 \\ S(t)-w(t,i,0) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,i+1,0) ] &~~\text{if}~~ 0<i<\bar{u}, t<T, \\ & ~~\text{and}~~ s=0, \\ S(t)-w(t,\bar{u},0) +\beta\left[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\bar{u},0)\right] &~~\text{if}~~ i=\bar{u}, t<T, ~~\text{and}~~ s\!=0 \\ S(t)-w(t,i,1) +\beta\left[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,-T_{in},1)\right] & ~~\text{if}~~-T_{in} \geq i, t<I,\\ & ~~\text{and}~~ s=1,\\ S(t)-w(t,-T_{in},1) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\underline{u},1) ] &~~\text{if}~~ -T_{in} = i, t=I, \\ & ~~\text{and}~~ s=1 \\ S(t)-w(t,i,1) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,i+1,1) ] &~~\text{if}~~ \underline{u}<i<\bar{u}, t<T,\\ & ~~\text{and}~~ s=1, \\ S(t)-w(t,\bar{u},1) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\bar{u},1)] & ~~\text{if}~~ i=\bar{u}, t<T_{end}, \\ & ~~\text{and}~~ s=1 \\ S(t)-w(T_{end},\bar{u},1) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(t+1,\bar{u},0) ] & ~~\text{if}~~ i=\bar{u}, t=T_{end}, \\ & ~~\text{and}~~ s=1 \\ S(t)-w(t,i,0) +\beta[\delta \Pi^{V} + (1-\delta)\Pi^{E}(T,\bar{u},0) ] &~~\text{if}~~ t \geq T, \end{array}\right. \end{aligned}}} $$

where match tenure beyond T no longer increases profits.

If a firm has a vacancy, the value of a vacancy is determined by the probability of meeting an unmatched worker, by the profits the firm expects to make from the match, and by the costs of posting a vacancy (ξ)

$$\Pi^{V}= - \xi + \beta \left[\sum_{i}q(i)\Pi^{E}(1,i,s) +\left(1-\sum_{i}q(i)\right)\Pi^{V}\right], $$

where s=1 if −T in i>−T out and s=0 otherwise.

Firms will post vacancies unless the expected profit from doing so is negative. Thus in the steady state equilibrium the number of firms in the economy will be determined by the condition that the expected profits from posting a vacancy are zero.42 Note that this also requires a free entry assumption.

8.4 B.4 Search technology

Assume there is no on-the-job search in the economy. The probability that a jobless individual receives a job offer depends on the probability the worker contacts a firm and the probability a firm has a vacancy.

8.4.1 B.4.1 Workers

The probability a firm has a vacancy is simply the total number of vacancies divided by the total number of firms

$$\frac{V}{F}. $$

If a firm has a vacancy, it will hire a worker and pay a wage which is the outcome of Nash bargaining between the worker and the firm, discussed in detail below. Applications for jobs arrive according to a Poisson process, where λ is the average number of applications filed by workers at each firm. It is further assumed that firms randomly draw workers from the applicant pool if there is more than one applicant.43 The probability a worker is offered a job is:

$$\frac{1-e^{-\lambda}}{\lambda}. $$

The conditional re-employment probabilities for unemployed workers and workers on income assistance can then be expressed as the product of the above components, multiplied by the worker’s search effort

$$m(i)=\frac{p(i)V}{\lambda F}\left(1-e^{-\lambda}\right), $$

where

$$\lambda=\frac{1}{F}\left(\sum_{i=1}^{\bar{u}}p(i)U(i)+\sum_{i=-T_{out}}^{0} p(i)A(i) \right). $$

Recall, p(i) are the contact probabilities of an individual in jobless state i. The contact probabilities are choice variables for the workers within the model and can be interpreted as search effort. Workers determine the optimal level of search effort by equating the marginal benefit from an increase in search effort with its marginal cost.44 The optimal level of search effort, for each labor market state and program eligibility combination, is described by:

$$p(i)= \left\{\begin{array}{ll} \left(\frac{\beta m(i)}{c_{u}z}\left[V^{E}(1,i-1,0)-V^{U}(i-1)\right]\right)^{ \frac{1}{z}},& i > 1 \\ \left(\frac{\beta m(i)}{c_{u}z}\left[V^{E}(1,i-1,0)-V^{A}(0)\right]\right)^{ \frac{1}{z}},& i = 1\\ \left(\frac{\beta m(i)}{c_{a}z}\left[V^{E}(1,i-1,0)-V^{A}(i-1)\right]\right)^{ \frac{1}{z}},& 0 \geq i > -T_{in} \\ \left(\frac{\beta m(i)}{c_{a}z}\left[V^{E}(1,i-1,1)-V^{A}(i-1)\right]\right)^{ \frac{1}{z}},& -T_{in} \geq i > -T_{out} \\ \left(\frac{\beta m(i)}{c_{a}z}\left[V^{E}(1,i-1,0)-V^{A}(-T_{out})\right]\right)^{\frac{1}{z}},& i = -T_{out} \\ \end{array} \right. $$

8.4.2 B.4.2 Firms

From the firm’s perspective, the probabilities of meeting potential workers from unemployment and welfare are the fraction of workers from unemployment and welfare that transit to employment, divided by the total number of vacancies

$$q(i)=\frac{m(i)U(i)}{V},\qquad \bar{u} \geq i > 0, \qquad \text{and} \qquad q(i)=\frac{m(i)A(i)}{V},\qquad 0 \geq i \geq -T_{out} $$

respectively.

8.5 B.5 Equilibrium wage determination

After meeting in the labor market, a firm and a worker bargain over wages by making alternating wage offers until both sides find the offer acceptable. It is assumed that both parties have equal bargaining power, but may have different threat points. The equilibrium of this game is the Nash cooperative bargaining solution and results in workers and firms splitting the surplus of a match evenly. The surplus of the match from the worker’s perspective is the difference between employment at the equilibrium wage and the worker’s outside option, which depends on their current labor market state and program eligibility, as well as on SSP supplement eligibility. The surplus from the perspective of the firm is the difference between the profits the firm receives at the equilibrium wage and the value of leaving the vacancy open. It is further assumed that the bargaining process is constrained such that the wage can not fall below the minimum wage \(\underline {w}\). The equilibrium wage is \(\max \{w(t,i,s), \underline {w}\}\), where w(t,i,s) solves

$$V^{E}(t,i,s)-V^{j}(i)=\Pi^{E}(t,i,s)-\Pi^{V}, $$

and where V j(i){V A(i),V U(i)} is the value of the outside option i. In the following section, we define the steady state conditions that govern the evolution of the economy.

8.6 B.6 Steady state conditions

Let E denote the steady state number of jobs occupied by workers and V the number of vacancies. By definition, the total number of jobs in the labor market is equal to the total number of occupied jobs and the total number of vacancies

$$F=E+V. $$

Denote the total number of individuals in the labor market by L. The total number of individuals can be decomposed into three groups. First the employed, who are distinguished by their current job tenure, their current outside option, and their SSP supplement eligibility status

$$E=\sum^{T}_{t=1} \sum_{i} \sum_{s} E(t,i,s)+\bar{E}, $$

where \(\bar {E}\) is the group of workers no longer experiencing on-the-job wage growth. The second group are on welfare and are distinguished by their welfare duration, which determines their SSP eligibility:

$$A=\sum_{i=-T_{out}}^{0} A(i). $$

The final group are unemployed individuals (U), who can remain unemployed for up to a maximum of \(\bar {u}\) periods

$$U=\sum^{\bar{u}}_{i=1}U(i), $$

where U(i) indicates the number of unemployed persons with i periods of benefits remaining.

The total number of individuals in the labor market can therefore be expressed as the sum of the above components

$$L=E+A+U. $$

Using the above definitions, we can describe the conditions governing the steady state, where the flows in and out of every labor force state must be equal over time. The steady state conditions for each state and eligibility combination are discussed in turn below.

8.6.1 B.6.1 Employment

As above, let m(i) denote the probabilities that jobless individuals from jobless state i match with a firm with a vacancy. The flow into the first period of employment includes those workers from welfare and unemployment who receive job offers. They are indexed by their respective outside options and their SSP eligibility as this will determine their progression of benefit entitlements. In subsequent periods, the inflow consists of workers who were employed in the previous period and who were not exogenously separated from their jobs

$$\begin{array}{@{}rcl@{}} E(1,i,0) &=& m(i)U(i), \quad \underline{u} \geq i > 0 \\ E(1,i,0) &=& m(i)A(i), \quad 0 \geq i > -T_{in} \\ E(1,i,1) &=& m(i)A(i), \quad -T_{in} \geq i > -T_{out} \\ E(1,i,0) &=& m(i)A(i), \quad i = -T_{out} \\ E(t,i,0) &=& (1-\delta)E(t-1,i-1,0), \quad 1 < t \leq T, i>0 ~~\text{and}~~ s=0 \\ E(t,i,1) &=& (1-\delta)E(t-1,i-1,1), \quad 1 < t < T_{end}, ~~\text{and}~~ s=1 \\ E(t,i,0) &=& (1-\delta)E(t-1,i-1,1), \quad t = T_{end}, ~~\text{and}~~ s=1 \\ \delta \bar{E} &=& (1-\delta)E(T,\bar{u}).\\ \end{array} $$

8.6.2 B.6.2 Welfare

The flow into the first period of Income Assistance includes those employed workers who were exogenously separated from their jobs and ineligible for unemployment benefits and unemployed workers no longer eligible for unemployment benefits. The flow into the first period of SSP eligibility includes those on welfare who have been on welfare long enough to qualify, plus those who were exogenously separated from jobs in which they were receiving the SSP supplement but not yet eligible for unemployment benefits. The flows into all other periods of income assistance consist of those workers in the previous period of income assistance who did not match with an employer. Finally, those individuals who did not find employment before SSP eligibility expired transit to A(−T out ), where the flow in and out of this state must be equal.

$$\begin{array}{@{}rcl@{}} \delta \sum_{t=1}^{I} E(t,0,0)+(1-m(1))U(1) &=& A(0) \\ \delta \sum_{t=1}^{I} E(t,-T_{in},1)+(1-m(-T_{in}+1))A(-T_{in}+1) &=& A(-T_{in})\\ (1-m(i+1))A(i+1)&=&A(i), \qquad 0>i>-T_{out}, \\ &&\qquad\qquad \text{and}~~ i\neq -{T_{in}}\\ (1-m(-T_{out}+1))A(-T_{out}+1) &=& m(-T_{out})A(-T_{out}). \end{array} $$

8.6.3 B.6.3 Unemployment

Employed workers who are separated from their jobs and who are eligible for the maximum months of unemployment benefits flow into the first period of unemployment, \(U(\bar {u})\). For U(i) where \(0 < i < {\bar {u}}\), the inflow consists of unemployed workers from the previous period who did not find jobs, and workers separated from their jobs who qualify for less than the maximum number of benefit months. All workers flow out of the unemployment state when benefits run out due to the time limitations in the unemployment insurance program

$$\begin{array}{@{}rcl@{}} \delta \sum_{t}E(t,\bar{u},0) &=& U(\bar{u})\\ \delta \sum_{t}\sum_{i}E(t,i,0)+ \delta \sum_{t\geq I}\sum_{i}E(t,i,1) &&\\ +(1-m(i+1))U(i+1)&=&U(i)~~\text{if}~~0<1<\bar{u}. \end{array} $$

8.7 B.7 Model solution

Solving the model for the control group entails solving a system of 145 equations and unknowns. The set of equations in the system includes:

  1. 1.

    steady state stocks and flows for unemployment, employment, and IA (12 equations)

  2. 2.

    the value functions for employment, IA, and unemployment (108 equations)

  3. 3.

    the re-employment probability from IA (1 equation)

  4. 4.

    the re-employment probability from UI (10 equations)

  5. 5.

    the number of applications per firm (λ) (1 equation)

  6. 6.

    the first order conditions for search intensity when on IA (1 equation)

  7. 7.

    the first order conditions for search intensity when on UI (10 equations)

  8. 8.

    the number of firms (F=E+V) (1 equation)

  9. 9.

    the vacancy rate (V=0.032F) (1 equation)

The set of unknowns includes:

  1. 1.

    the number of individuals unemployed with i months of benefits (10 unknowns)

  2. 2.

    the values of being in each state, V A, V U(i), V E(t,i) (108 unknowns)

  3. 3.

    the re-employment probabilities, m(i) (11 unknowns)

  4. 4.

    the search intensities, p(i) (11 unknowns)

  5. 5.

    the average number of applications filed by workers at each firm λ (1 unknown)

  6. 6.

    the number employed, E (1 unknown)

  7. 7.

    the number on IA, A (1 unknown)

  8. 8.

    the number of firms, F (1 unknown)

  9. 9.

    the number of vacancies, V (1 unknown)

Solving the model for the program group entails solving the additional set of 72 equations:

  1. 1.

    the value functions for being employed and receiving the SSP top up (36 equations)

  2. 2.

    the value functions for being on IA and treated with the SSP offer (12 equations)

  3. 3.

    the re-employment probabilities when on IA with the SSP offer (12 equations)

  4. 4.

    the first order conditions for search effort when on IA with the SSP offer (12 equations),

and 72 unknowns:

  1. 1.

    the value of being employed and receiving the SSP top up (36 unknowns)

  2. 2.

    the value of being on IA and treated with the SSP offer (12 unknowns)

  3. 3.

    the re-employment probabilities (12 unknowns)

  4. 4.

    optimal search effort (12 unknowns).

The model has no closed form solution. To see how we solve the model, define a 145×1 vector for the unknowns, x, and a system of non-linear equations, f(x). We solve this system of equations in two steps. We obtain starting values by solving

$$x=\arg \min f(\mathbf{x})'f(\mathbf{x})</p><p class="noindent">$$

using a Quasi-Newton method, where the Hessian is updated using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update. Second, using the above starting values, we solve the system of non-linear equations using the secant method (Broyden’s approximation to Jacobian).45 The solution to the model directly yields the simulated version of the model.

References

  • Albrecht, J, van den Berg G, Vroman S (2009) The aggregate labor market effect of the swedish knowlege lift program. Rev Econ Dyn12: 129–146.

    Article  Google Scholar 

  • Alvarez, F, Veracierto M (2000) Labor-market policies in an equilibrium search model In: NBER Macroeconomics Annual 1999, 265–316.. MIT, Cambridge, MA.

    Google Scholar 

  • Attanasio, O, Meghir C, Santiago A (2012) Education choices in mexico: Using a structural model and a randomized experiment to evaluate progresa. Rev Econ Stud 79: 37–66.

    Article  Google Scholar 

  • Auspos, P, Miller C, Hunter JA (2000) Final Report on the Implementation and Impacts of the Minnesota Family Investment Program in Ramsey County. Manpower Demonstration Research Corporation, New York.

    Google Scholar 

  • Bajari, P, Hortaçsu A (2005) Are structural estimates of auction models reasonable? evidence from experimental data. J Polit Econ 113(4): 703–741.

    Article  Google Scholar 

  • Bitler, M, Gelbach J, Hoynes H (2008) Distributional impacts of the self-sufficiency project. J Public Econ 92: 748–765.

    Article  Google Scholar 

  • Bos, JM, Huston AC, Granger RC, Duncan GJ, Brock TW, McLoyd VC, et al. (1999) New Hope for People with Low Incomes: Two-Year Results of a Program to Reduce Poverty and Reform Welfare. Manpower Demonstration Research Corporation, New York.

    Google Scholar 

  • Cahuc, P, Le Barbanchon T (2010) Labor market policy evaluation in equilibrium: Some lessons of the job search and matching model. Labour Econ 17: 196–205.

    Article  Google Scholar 

  • Card, D, Hyslop DR (2005) Estimating the effects of a time-limited earnings subsidy for welfare-leavers. Econometrica73(6): 1723–1770.

    Article  Google Scholar 

  • Christensen, B. J., Lentz R., Mortensen D. T., Neumann G. R., Werwatz A. (2005) On-the-job search and the wage distribution. J Labor Economics 23(1): 31–58.

    Article  Google Scholar 

  • Cole, HL, Rogerson R (1999) Can the mortensen-pissarides matching model match the business-cycle facts?Int Econ Rev 40(4): 933–959.

    Article  Google Scholar 

  • Connolly, H., Gottschalk P. (2009) Do earnings subsidies affect job choice? the impact of ssp subsidies on job turnover and wage growth. Can J Econ 42(4): 1276–1304.

    Article  Google Scholar 

  • Crépon, B, Duflo E, Gurgand M, Rathelot R, Zamora P (2013) Do labor market policies have displacement effects? evidence from a clustered randomized experiment. Q J Econ 128: 531–580.

    Article  Google Scholar 

  • Davidson, C, Woodbury SA (1993) The displacement effect of reemployment bonus programs. J Labor Econ 11(4): 575–605.

    Article  Google Scholar 

  • den Haan, WJ, Ramey G, Watson J (2000) Job destruction and propagation of shocks. Am Econ Rev 90(3): 482–498.

    Article  Google Scholar 

  • Diamond, PA (1982) Aggregate demand management in search equilibrium. J Polit Econ 90(5): 881–94.

    Article  Google Scholar 

  • Ferrall, C (2012) Explaining and forecasting results of the self-sufficiency project. Rev Econ Stud 79(4): 1495–1526.

    Article  Google Scholar 

  • Foley, K, Schwartz S (2003) Earnings supplements and job quality among former welfare recipients: Evidence from the self-sufficiency project. Relations Industrielles/Industrial Relations 58: 258–286.

    Article  Google Scholar 

  • Ford, RDG, Foley K, Tattrie D, Jimenez L (2003) Can Work Incentives Pay for Themselves?: Final Report on the Self-Sufficiency Project for Welfare Applicants. Social Research and Demonstration Corporation, Ottawa, Canada.

    Google Scholar 

  • Galarneau, D, Krebs H, Morissette R, Zhang X (2001) The Quest for Workers: A New Portrait of Job Vacancies in Canada. Statistics Canada, Ottawa, Canada.

    Google Scholar 

  • Gautier, P, Muller P, van der Klaauw B, Rosholm M, Svarer M (2012) Estimating equilibrium effects of job search assistance. IZA Discussion Paper No. 6748, Bonn, Germany.

  • Gertler, M, Trigari A (2009) Unemployment fluctuations with staggered nash wage bargaining. J Polit Econ 117: 38–86.

    Article  Google Scholar 

  • Hall, R (2005) Employment fluctuations with equilibrium wage stickiness. Am Econ Rev 95(1): 50–65.

    Article  Google Scholar 

  • Hansen, LP, Heckman JJ (1996) The empirical foundations of calibration. J Econ Perspect 10(1): 87–104.

    Article  Google Scholar 

  • Hopenhayn, H, Rogerson R (1993) Job turnover and policy evaluation: a general equilibrium analysis. J Polit Econ 101(5): 915–938.

    Article  Google Scholar 

  • Hornstein, A, Krusell P, Violante G (2011) Frictional wage dispersion in search models: A quantitative assessment. Am Econ Rev 101: 2873–2898.

    Article  Google Scholar 

  • Jovanovic, B (1979) Job matching and the theory of turnover. J Polit Econ 87: 972–990.

    Article  Google Scholar 

  • Jovanovic, B (1984) Matching, turnover, and unemployment. J Polit Econ 92: 108–122.

    Article  Google Scholar 

  • Judd, KL (1998) Numerical Methods in Economics. MIT press, Cambridge, MA.

    Google Scholar 

  • Kamionka, T, Lacroix G (2008) Assessing the external validity of an experimental wage subsidy. Annales d’Economie et de Statistique91/92: 357–384.

    Google Scholar 

  • Keane, MP, Wolpin KI (2007) Exploring the usefulness of a nonrandom holdout sample for model validation: Welfare effects on female behavior. Int Econ Rev 48(4): 1351–1378.

    Article  Google Scholar 

  • Kydland, FE, Prescott EC (1996) The computational experiment: an econometric tool. J Econ Perspect 10(1): 69–85.

    Article  Google Scholar 

  • Lin, W, Robins P, Card D, Harknett K, Lui-Gurr S (1998) When Financial Incentives Encourage Work: Complete 18-month Findings from the Self-Sufficiency Project: Executive Summary. Social Research and Demonstration Corporation, Ottawa, Canada.

    Google Scholar 

  • Lise, J, Robin J-M (2014) The macro-dynamics of sorting between workers and firms. IFS Working PaperW13(/22)London, UK.

  • Lise, J, Seitz S, Smith J (2004) Equilibrium policy experiments and the evaluation of social programs. NBER working paper: 10283Cambridge, MA.

  • Ljungqvist, L, Sargent TJ (1998) The european unemployment dilemma. J Polit Econ 106: 514–550.

    Article  Google Scholar 

  • McCall, JJ (1970) Economics of information and job search. Q J Econ 84(1): 113–126.

    Article  Google Scholar 

  • Michalopoulos, C, Robins P, Card D (2005) When financial work incentives pay for themselves: Evidence from a randomized social experiment for welfare recipients. J Public Econ 89: 5–29.

    Article  Google Scholar 

  • Michalopoulos, C, Tattrie D, Miller C, Robins P, Morris P, Gyarmati D, Redcross C, Foley K, Ford R (2002) Making Work Pay: Final Report on the Self-Sufficiency Project for Long-Term Welfare Recipients. Social Research and Demonstration Corporation, Ottawa, Canada.

    Google Scholar 

  • Mortensen, DT (1982) Property rights and efficiency in mating, racing, and related games. Am Econ Rev 72(5): 968–79.

    Google Scholar 

  • Mortensen, DT, Nagypal E (2007) More on unemployment and vacancy fluctuations. Rev Econ Dyn 10: 327–47.

    Article  Google Scholar 

  • Mortensen, DT, Pissarides CA (1994) Job creation and job destruction in the theory of unemployment. Rev Econ Stud 61(3): 397–415.

    Article  Google Scholar 

  • Pissarides, CA (1985) Short-run equilibrium dynamics of unemployment, vacancies, and real wages. Am Econ Rev 75(4): 676–90.

    Google Scholar 

  • Pissarides, CA (2000) Equilibrium Unemployment Theory. 2nd ed.. The MIT Press.

  • Plesca, M (2010) A general equilibrium analysis of the employment service. J Hum Capital 4: 274–329.

    Article  Google Scholar 

  • Riddell, C, Riddell C (2014) The pitfalls of work requirements in welfare-to-work policies: Experimental evidence on human capital accumulation in the self-sufficiency project. J Public Econ 117: 39–49.

    Article  Google Scholar 

  • Rogerson, R, Shimer R, Wright R (2006) Search-theoretic models of the labor market-a survey. J Econ Lit 43: 959–988.

    Article  Google Scholar 

  • Shimer, R (2005) The cyclical behavior of equilibrium unemployment and vacancies. Am Econ Rev 95(1): 25–49.

    Article  Google Scholar 

  • Sianesi, B (2014) Dealing with randomisation bias in a social experiment: the case of era. IFS Working PapersW14(/10). London, UK.

  • Todd, PE, Wolpin KI (2006) Assessing the impact of a school subsidy program in mexico: Using a social experiment to validate a dynamic behavioral model of child schooling and fertility. Am Econ Rev 96(5): 1384–1417.

    Article  Google Scholar 

  • Wise, DA (1985) A behavioral model versus experimentation: The effects of housing subsidies on rent. Methods Oper Res 50: 441–89.

    Google Scholar 

  • Yashiv, E (2006) Evaluating the performance of the search and matching model. In: Bunzel H, Christensen BJ, Neumann GR, Robin J-M (eds)Structural Models of Wage and Employment Dynamics:Volume 275 of Contributions to Economic Analysis, Chapter 20, 509–550.. Elsevier, Amsterdam.

    Chapter  Google Scholar 

  • Zabel, J, Schwartz S, Donald S (2013) An analysis of the impact of the self-sufficiency project on wages. Empirical Economics 44(1): 231–259.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank John Ham, José-Víctor Ríos-Rull, Doug Tattrie, the editor, and two anonymous referees for many helpful comments. Financial support from the Social Research and Demonstration Corporation is gratefully acknowledged. All empirical analyses in this paper were performed prior to September 30, 2007. The micro-data from the Self-Sufficiency Project are available within Canada Research Data Centres through a research contract with Statistics Canada.

Responsible editor: Pierre Cahuc.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jeffrey Smith.

Additional information

Competing interests

The IZA Journal of Labor Economics is committed to the IZA Guiding Principles of Research Integrity. The authors declare that they have observed these principles.

Rights and permissions

Open Access  This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lise, J., Seitz, S. & Smith, J. Evaluating search and matching models using experimental data. IZA J Labor Econ 4, 16 (2015). https://doi.org/10.1186/s40172-015-0031-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40172-015-0031-7

Keywords