Multiple job holding, local labor markets, and the business cycle

Abstract

About 5 % of US workers hold multiple jobs, which can exacerbate or mitigate employment changes over the business cycle. Theory is ambiguous and prior literature is not fully conclusive. We examine the relationship between multiple job holding and local unemployment rates using a large Current Population Survey data set of workers in urban labor markets during 1998–2013. Labor markets with high unemployment have moderately lower rates of multiple job holding. Yet no relationship between multiple job holding and unemployment is found within markets over time, with near-zero estimates being precisely estimated. Multiple job holding is largely acyclic.

JEL Classification: J21

1 Introduction

Roughly 5 % of US workers hold multiple jobs. The common perception among journalists and the public is that multiple job holding is countercyclical, used by workers to help offset household income losses during a recession. Numerous news stories highlight anecdotal evidence of individuals taking multiple jobs when economic conditions are poor.¹ Neither theory nor prior evidence, however, provides an unambiguous answer to the question of whether multiple job holding is procyclical or countercyclical. Based on labor supply theory, workers’ willingness to hold multiple jobs may be countercyclical if household income effects are strong. Desire for a second job can arise from losses in work hours or other earnings on one’s primary job or from an earnings loss by another household member. Even if labor supply for multiple jobs is countercyclical, product and labor demand falls in a recession, resulting in fewer opportunities to hold multiple jobs absent extreme wage flexibility. The limited evidence that exists on multiple job holding and the business cycle is suggestive, but not fully conclusive.

In this paper, we use a large micro data set for 1998–2013 in order to examine how multiple job holding varies across and within US labor markets (MSAs) with respect to local unemployment and employment growth rates. In addition to examining how overall multiple job holding varies with respect to local labor market conditions, we address several related questions. Among these are how multiple job holding over the cycle has varied over time and among demographic groups, how it varies for workers whose primary jobs are salaried versus hourly, and whether responses to business conditions are symmetric with respect to increases versus decreases in unemployment. We briefly address the question of how closely second jobs match primary jobs and whether differences between the two, measured by occupational indices of skill-related job tasks and working conditions, vary over the business cycle.

2 Why do workers hold multiple jobs?

Although our focus is on how multiple job holding varies with the business cycle, it is helpful to discuss the reasons for holding multiple jobs and the type of workers most likely to do so. Such a discussion helps inform our understanding of how multiple jobs might vary with macroeconomic conditions.

Most explanations for taking multiple jobs can be grouped into one of two categories, either because of an “hours constraint” or in order to obtain a preferred “job portfolio.” An hours constraint on a worker’s principal (say, highest wage) job can readily explain why a worker might increase utility by taking a second job, even at a lower wage. Alternatively, a worker not facing an hours constraint on the first job may take a higher paying second job that does have constrained hours, say, a temporary job or limited hours per week. Roughly 45 % of US wage and salary workers are salaried rather than paid by the hour. Although salaried jobs generally do not have an explicit hours constraint, they do have an “earnings constraint” that can work in much the same way, leading some salaried workers to take a second job in order to increase their earnings. We have not seen this argument made in prior literature.

A “job portfolio” framework provides several explanations for multiple job holding. A worker may simply prefer diversity in job tasks, being happier dividing time in two different jobs or occupations.² In this spirit, Renna and Oaxaca (2006) develop a job portfolio model based on a personal preference for job differentiation. Alternatively, workers may choose to work a second job as a form of insurance (i.e., diversifying one’s human capital) due to a high level of employment or income uncertainty in a first job. Or workers wanting to switch occupations or employers due to a poor match may use a second job as a source of on-the-job training that will facilitate a utility-enhancing move. Finally, a worker may choose to work at a second job for financial or family circumstances that are temporary, at the same time expecting that the current primary job offers the best long-run job match. Using panel data, Panos et al. (2014) provide interesting evidence on skill diversification and mobility among British dual job holders.

Given the varied reasons for which individuals hold multiple jobs, predicting how multiple job holding differs across demographic groups can be problematic. Hipple (2010) provides descriptive evidence on multiple job holding rates for employed workers for 1994 through 2009. Primary job occupations with particularly high rates of dual job incidence among men are firefighters, emergency medical technicians and paramedics, and teachers and among women are dental hygienists, psychologists, teachers, and therapists.³ Some of these occupations (e.g., teachers) are salaried but earnings constrained, with hours that are either sufficiently predictable or flexible as to allow a second job. Those holding multiple jobs in 2009 worked an average 11 h per week more than did single job holders (Hipple 2010). In surveys asking why individuals hold multiple jobs (last asked in the May 2004 CPS), economic reasons are predominant, although roughly one in five cites enjoyment of the second job. In subsequent analysis, we present not only descriptive evidence on how multiple job holding differs among various worker groups but also multivariate analysis providing evidence on how worker, job, and location attributes are associated with holding multiple jobs.

3 Multiple job holding and unemployment

In our discussion and subsequent analysis of multiple jobs, we largely follow the US Bureau of Labor Statistics (BLS) in how multiple jobs are defined (see Hipple 2010). The BLS defines a multiple job holder as an individual who (a) holds wage and salary jobs with two or more employers during the survey reference week, (b) combines a wage and salary job with self-employment, or (c) combines a wage and salary job with one as an unpaid family worker. In our subsequent empirical work, we include only those multiple job holders whose primary job is a wage and salary job.⁴

The Current Population Survey (CPS) began regularly collecting information on multiple job holding in 1994 as part of the survey’s major redesign (prior to 1994, occasional CPS supplements included information on multiple job holding). Figure 1a, b shows the BLS national monthly rates of multiple job holding and unemployment for the period January 1994 through December 2014, shown separately for men and women. Each rate is computed as a 3-month average centered on the designated month (the exception is January 1994, which is the average rate for January and February). Neither series is seasonally adjusted. An obvious characterization of the national multiple job holding series is that it varies little with respect to changes in unemployment, but has shown a modest downward trend over time, more so for men than for women. The 1994–2014 period included a strong boom in the late 1990s and two recessions, the most recent being quite large and lengthy. There appears to be no obvious, substantive relationship between national unemployment and multiple job holding rates over time. Although there is limited seasonal variation in multiple job holding (Hipple 2010), our subsequent regression analysis includes month fixed effects, the magnitude of which are relatively small.

Fig. 1

a Monthly nationwide multiple job holding rates and unemployment rates, men, 1994–2014. b Monthly nationwide multiple job holding rates and unemployment rates, women, 1994–2014

There is a reasonably robust literature exploring the determinants of multiple job holding, but little on how it varies over the cycle.⁵ CPS data, much of it from an earlier era, have been used by others to examine the incidence of multiple job holding (Stinson 1987, 1990; Kimmel and Powell 1999). The primary advantage of CPS data is that samples are large and nationally representative.⁶ The 1979 National Longitudinal Survey of Youth (NLSY79) and the Survey of Income and Program Participation (SIPP) are relatively well suited for identifying and measuring the duration of multiple jobs (Amuédo-Dorantes and Kimmel 2009; Kimmel and Conway 2001), but each has modest sample sizes. In the Panel Study of Income and Dynamics (PSID), it is possible to identify more than one job but not always possible to discern whether or not the jobs are held simultaneously (Paxson and Sicherman 1996; Averett 2001).

The literature on multiple job holding and cyclicality is neither large nor fully conclusive. Amuédo-Dorantes and Kimmel (2009) provide a nice summary of this literature, identifying studies that report evidence of countercyclical, acyclic, and procyclical relationships. These authors’ analysis of NLSY79 data uses state employment growth as a measure of the business cycle. They conclude that multiple job holding among men is largely acyclic, while female multiple job holding switched from countercyclical during the 1980s and early 1990s to procyclical by 2000–2002.

A recent paper by Lalé (2015) provides detailed descriptive evidence of worker transitions in the CPS in order to better understand the gradual decline in multiple job holding over time. He shows that lower rates are strongly associated with declining monthly transitions from single job to multiple job holding. Although the focus of his analysis is not on the business cycle, he concludes there is little evidence of a substantive relationship between multiple job holding and the national business cycle in time series patterns (such as in Fig. 1) showing aggregate US data (Lalé 2015, p. 2). In contrast to prior literature on multiple job holding and the business cycle, our analysis focuses on how local area multiple job holding varies with respect to local labor market business conditions.

A recent working paper by Zangelidis (2014) examining European evidence on multiple job holding is the paper most comparable to our work. As do we, he uses a large micro-level data set, in his case the European Union Labour Force Survey (EU-LFS) for 1998–2011. Across the 28 EU countries, the multiple job holding (MJH) rate is much lower than that in the USA, 3.2 % in the EU versus about 5 % in the USA. That said, there is large variability in the MJH rate across the EU countries (rates range from less than 1 to 9 %), although mean weekly hours on the second job vary little across countries, averaging 12.9 h across the continent. Most relevant to our work, Zangelidis finds an overall procyclical MJH pattern using country unemployment rates as the primary business cycle measure. This matches our findings for the USA, absent labor market fixed effects. Zangelidis does not focus on within versus across labor market differences in MJH responsiveness to the business cycle, although his analysis does include country fixed effects. Zangelidis introduces the novel concept of second job intensity, measured by the percentage of total work hours due to the second job. Among multiple job holders, the intensity measure averages 26.7 % across all the 28 countries and displays limited variation (the values range from 22 to 34 %). The intensity measure of second job holding varies procyclically.⁷

Our analysis can be differentiated from prior literature in several ways. We use an unusually large US micro data set covering a lengthy time period (1998–2013), enabling us to obtain precise estimates of the effects of labor market conditions on MJH. Because only one-in-twenty workers hold multiple jobs, sample size matters. Our large sample enables us to examine differences in MJH responsiveness to business conditions for different worker groups based on gender, marital status, foreign born, and hourly versus salaried workers. And we are able to examine whether MJH responsiveness to business conditions changed during the Great Recession (and over time more generally). A key contribution of the paper is our focus on local labor markets (MSAs), distinguishing between estimates of MJH cyclicality that rely on cross-labor market differences versus within labor market changes in business conditions. We examine differences in how MJH is related to the two business cycle measures used in prior literature, the unemployment rate and employment growth. We find that the two measures are weakly correlated across MSAs and have largely independent effects on MJH. Finally, we utilize measures of occupational job attributes, which enable us to construct occupational skill and working condition indices for the primary and secondary jobs.

4 How should MJH vary over the business cycle?

Even if workers are more likely to want to hold multiple jobs during downturns, it need not follow that MJH will increase—creation of jobs requires employer demand as well as labor supply. In what follows, we examine how MJH varies with respect to two alternative business cycle measures, the unemployment rate and growth rate in employment. We expect the two measures to exhibit similar relationships with MJH, albeit with opposite signs.⁸ That said, area unemployment and employment growth rates are weakly correlated in the USA (Rappaport 2012).

Each measure has strengths and weaknesses. The unemployment rate is a relatively direct measure of labor market tightness and the business cycle. However, local area unemployment rates are based on relatively small samples of households, supplemented with administrative data on employment and unemployment benefit claims. Our measure of employment growth, from the Quarterly Census of Employment and Wages (QCEW), provides a less noisy measure but is conceptually a less direct measure of the business cycle since growth is heavily influenced by factors unrelated to cyclicality. Moreover, there is a mechanical correlation between MJH and employment growth measured with establishment data. Unlike household surveys in which employment totals are based on persons and not jobs (i.e., a dual job holder is counted as one worker rather than two), establishment surveys measure employment based on the number of jobs; hence, a dual job holder would be counted twice. Measures of employment growth (measured as a percentage or log difference) are not affected as long as the rate of MJH is constant over time, but a small positive (negative) correlation between MJH and employment growth is mechanically generated if the MJH rate increases (decreases). We also provide supplemental analysis for the larger MSAs examining how MJH is related to measures of labor market vacancies (measured by the number of help wanted ads).

Theory suggests that income and substitution effects have opposite effects on the labor supply for multiple jobs. During a recession, income effects should lead to increased desire for second jobs due to earnings losses on the primary job (due to reductions in hours and/or wage reductions). Or it may result from a job or earnings loss of another household member. The reverse is true during an expansion. Substitution effects work in the opposite direction, with a weak (strong) economy lowering (raising) wage offers in second jobs.

In measuring labor supply for multiple jobs, we observe whether workers hold multiple jobs, but do not observe whether workers with a single job seek a second job. Hence, one cannot directly observe the labor supply for multiple jobs (conditional on the wage) absent the unrealistic assumption of market clearing. And it makes little sense to assume that markets clear if one is studying employment over the business cycle. Hence, our empirical work can identify the relationship between MJH and the business cycle, but the observed relationship reflects labor demand and job opportunities as well as labor supply preferences. Even if labor supply for multiple jobs is countercyclical, this need not result in increased MJH during a recession. That said, we provide supplementary evidence showing that among married couples, income shocks from changes in husbands’ employment affects wives’ MJH status. We see minimal response in husbands’ MJH behavior following changes in wives’ labor force status.

Of particular interest in the paper, given our large sample sizes, is how MJH responsiveness to the business cycle might differ across worker groups, for example between women and men and for those in hourly versus salaried jobs. Women have moderately higher MJH rates than do men. Unemployment for men tends to be more cyclical than for women, so one might expect employed married women to have a less cyclical MJH response than do married men. During a recession with large male job losses, we would expect negative income effects and increased husbands’ time at home to increase married women’s willingness to take a second job, despite such jobs being difficult to find. We have no priors for differences between single men and women.

Unexplored in prior studies are possible differences between those in hourly and those in salaried jobs. Weekly earnings in hourly jobs are likely to be more cyclical than earnings in salaried jobs. Earnings in hourly jobs vary due to changes in hours worked and in the marginal wage for overtime hours (1.5 W vs. W). Hence, hourly workers may have stronger income effects and could exhibit less cyclical or more countercyclical dual job labor supply. If labor demand is more cyclical for hourly than for salaried jobs, however, those holding hourly first jobs may exhibit more rather than less cyclicality than holders of salaried primary jobs.

5 Data

The goal of this paper is to identify how multiple job holding (MJH) responds to the business cycle and local labor market conditions. To examine this, we use CPS Monthly Outgoing Rotation Group (MORG) data from January 1998 through December 2013 (questions about multiple jobs were added to the CPS in 1994). Each month, the CPS—conducted by the US Census Bureau—interviews about 60,000 households, collecting a variety of information about labor market behavior, demographics, and family characteristics. Households are in the survey for a total of 8 months: they are interviewed for 4 consecutive months (rotation groups 1–4), then out of the survey the next 8 months, and then reenter the survey the following 4 months (rotation groups 5–8). We use the outgoing rotation group files (groups 4 and 8), the quarter sample each month that provides information on usual weekly earnings and hours worked, along with additional information on multiple jobs (see Table 1).

Table 1 Means for single and multiple job workers, selected variables

In the CPS, all employed respondents have been asked the following question: “Last week, did you have more than one job (or business), including part-time, evening, or weekend work?” If they answer “yes,” they are then asked how many jobs (or businesses) they had altogether and how many hours they worked each week at all their jobs. The primary job is defined as the one with the most hours worked. For workers holding three or more jobs, information is recorded only for the primary and second jobs, the latter being the job at which they work the second-greatest number of hours.⁹ For both the primary and second jobs, information is collected on class of the job (private for-profit, private not-for-profit, federal, state, or local), detailed industry and occupation, and usual weekly hours worked. Earnings information is available for the MORG quarter sample, but only for the primary job.

Our primary estimation sample includes 1,850,757 non-student wage and salary workers (on their primary job), ages 18–65, for 1998 through 2013, located in 258 MSAs throughout the USA (this accounts for roughly three quarters of the US workforce).¹⁰ In this sample, the (unweighted) MJH rate is 5.0 %, with 1,757,547 single job holders and 93,210 multiple job holders (few of these hold more than two jobs).¹¹ As explained when we discuss Table 1, weighting lowers the MJH rate to 4.7 %. Workers self-employed in their primary job but with a wage and salary second job are counted by BLS as multiple job holders, but we exclude them from our sample given that earnings (and other) information is not provided for their self-employment job. This large national sample of workers over 16 years provides us with substantial statistical power and the ability to examine differences both across labor markets and over time.

Workers in the CPS-MORG files are in the survey only once within a calendar year but typically appear in the survey for 2 consecutive years, assuming they remain in the same residence. Thus, it is possible to create short panels with two observations on each worker, 1 year apart, for up to half of the respondents in any given year’s survey. We provide supplementary analysis using CPS panels of worker-year pairs for the years 1998/1999 through 2012/2013.¹² The panels provide a robustness check on our principal cross section results, examining how 1-year changes in the unemployment rate within labor markets affect worker-specific transitions into and out of multiple jobs. These panels also enable us to examine how the MJH of husbands and wives are affected by changes over the past year in their spouse’s employment status.

Our unit of analysis is the individual worker, with emphasis given to MSA variation in business conditions (the unemployment rate and employment growth) across and within labor markets over time. Within labor market analysis is achieved by including MSA fixed effects in our estimating equations or, alternatively, using panel data to observe worker-specific changes in MJH over time. We find that much of the apparent cyclical response seen for MJH reflects differences across labor markets, while there is minimal cyclical response within labor markets over time. We use MSA unemployment, averaged of over 3 months (to reduce measurement error), as our principal measure of the business cycle.¹³ In work not shown, the use of a lagged unemployment measure produced results similar to those shown. In addition, we provide parallel analysis using employment growth as a measure of labor market tightness and business conditions.

For the analysis of occupational mismatch, we combine the CPS worker sample with data from the Occupational Information Network (O*NET), produced by the US Department of Labor’s Employment and Training Administration. This database is a comprehensive system for collecting, organizing, and describing data on job characteristics within occupations. We use O*NET 12.0, released in June 2007, and created a data set with O*NET occupational job descriptors.¹⁴ The O*NET indices were created based on SOC occupation codes used in the CPS beginning in 2003 and ending in 2010. Hence, our (limited) analysis using O*NET is based on a 2003–2010 CPS sample, whereas our primary analysis uses a larger CPS sample for 1998–2013.

Combining O*NET with the CPS allows us to account more directly for occupational skill requirements, job tasks, and working conditions for both primary and secondary jobs. Because the skill index is a strong correlate of wages, and wages are not reported in the CPS for workers’ second jobs, comparison of the skill index for the primary and secondary jobs provides information on likely differences in wages between primary and secondary jobs.

As described in Hirsch and Schumacher (2012), 206 O*NET job descriptors are used. Indices of occupational skills/tasks (SK) and physical working conditions (WC) are constructed using factor analysis. The SK factor index provides a linear combination of the 168 skill and task attributes, and the WC index provides linear combinations of 38 physical working condition attributes. The factor loadings of the SK first factor can be characterized as an index of occupational job tasks and cognitive skills of workers needed to successfully perform a job. SK accounts for 41 % of the total covariance among the 168 skill/task attributes across 501 census occupations. SK heavily loads O*NET measures of critical thinking, judgment and decision making, monitoring, written expression, speaking, writing, active listening, written comprehension, active learning, negotiation, and persuasion. WC accounts for 56 % of the total covariance among the physical working conditions and heavily loads extreme working conditions (e.g., temperature, lighting, contaminants, hazards) and strength requirements. The factor analysis is weighted by occupational employment from a large CPS sample.¹⁵

6 Descriptive evidence

Descriptive evidence on the merged CPS-O*NET database is provided in Table 1. Sample weights are used to estimate both the means and subsequent regression estimates. Weights have a substantive effect on overall MJH rates and, to a lesser extent, some of our other variables. While the weighted mean of MJH for our urban sample is 4.7 %, the unweighted mean is 5.0 %. The explanation for the difference is straightforward. MJH is substantially lower in large metropolitan areas than in smaller cities and in rural areas, while CPS sample weights are larger (smaller) in highly (less) populated areas due to relative under (over) sampling. Hence, unweighted sample means overstating the population MJH rate. As compared to the weighted mean of 4.7 % for our urban sample, the weighted MJH rate for rural areas and the small metro areas not identified in the CPS is 6.0 %. Weighting has a minimal effect on regression coefficients.

As seen in Table 1, usual weekly hours on the primary job is lower for multiple versus single job holders (37.2 vs. 39.8 h). Usual hours per week on a second job is 14.3 h, so dual job holders work substantially more hours (11.7) on average than do single job holders. Average hourly earnings in the primary job are roughly similar for single and multiple job holders, $22.62 and $22.92, respectively, in annual 2013 dollars. Multiple job holders tend to be more educated, having higher proportions of BA and graduate degrees than do single job holders. The proportion of employees who are hourly rather than salaried workers is similar for single and multiple job holders (56 and 58 %, respectively).

The descriptive data provide limited insight into the central question being asked in this paper. Mean MSA unemployment rates for the single and multiple job samples are highly similar (6.1 and 5.9 %). The lower unemployment rate for multiple job holders suggests MJH might be cyclical (i.e., vary inversely with unemployment), but the difference is too small to draw any conclusion.

Comparison of means for the occupational skill and working condition indices (SK and WC) shows that on the primary job, multiple job holders display a higher value of occupational skill (0.21 vs. 0.07) and slightly less demanding working conditions (−0.17 vs. −0.05) than do single job holders. Restricting the sample to dual job holders, one sees that these workers’ second jobs involved a noticeably lower level of skill (−0.08 vs. 0.21, a 0.3 s.d. difference) and slightly more demanding working conditions (−0.11 vs. −0.17). In our sample, 15.7 % of multiple job holders work in the same detailed occupation in their primary and second jobs (not shown in Table 1). Subsequent analysis examines how these skill and working condition differences vary with the business cycle. In wage analysis not shown, the coefficient on the SK skill index in a dense Mincerian log wage equation is about 0.20. Using that estimate, the 0.29 difference in the skill index between multiple job holders’ primary and secondary jobs (0.21 vs. −0.08) suggests a roughly 6 % wage advantage in the primary job (0.2 times 0.29).

7 Empirical estimation: models

To examine the response of MJH over the business cycle, linear probability models of MJH are specified (probit estimates at the sample means are highly similar). That is, we estimate

$$ {M}_{ikt}=\beta\;{x}_{ikt}+\theta {\mathrm{LM}}_{kt}+\gamma\;{U}_{kt}+\varepsilon, $$

(1)

where M _ikt represents the probability of individual i in labor market k in time period t holding multiple jobs, conditional on being employed as a wage and salary worker. Our principal measure of the business cycle is the monthly unemployment rate U (averaged over 3 months) in metropolitan area k during month t.¹⁶ An estimate of γ < 0 would indicate that MJH is procyclical, γ = 0 acyclic, and γ > 0 countercyclical. We subsequently provide analysis including employment growth as an alternative measure of the business cycle.

The vector x _ikt includes demographic, human capital, workplace (the primary job), and time period controls. These include indicator variables for education (5 dummies for 6 categories), age (9), gender, marital status (2), preschool and school-age children in the household (2), foreign-born citizen and non-citizen (2), public employment, union member, hours on primary job (5), industry (8) and occupation (15) in the primary job, and month and year dummies.¹⁷ The vector LM _kt represents labor market (MSA) characteristics other than the unemployment rate. We first estimate a specification without LM controls, then add region and MSA size dummies, and then MSA fixed effects (absorbing the region and size dummies). Standard errors are clustered by MSA. The purpose of the increasingly detailed location dummies is to move from estimates of γ based heavily on cross-sectional differences in multiple job outcomes to estimates based primarily on temporal changes in multiple job behavior within labor markets.

As previously discussed, supply-side forces likely lead to increases in the desire for MJH during downturns as workers respond to lower household income and increased financial risk. Yet demand-side forces may lead to MJH declines during downturns as employers provide fewer jobs. Net effects may be weak. Further ambiguity arises from possible asymmetry in the responsiveness of MJH to economic expansions and contractions, a possibility we have not seen considered in prior literature. To test for asymmetric responsiveness, we estimate a model that permits unemployment-level coefficients to differ depending on whether the unemployment rate has increased or decreased. We examine whether response differs for unemployment increases versus decreases. The estimated model is

$$ {M}_{ikt}=\beta\;{x}_{ikt}+\theta\;{\mathrm{LM}}_{kt}+{\gamma}^U\;\mathrm{UP}\cdot {U}_{kt}+{\gamma}^D\;\mathrm{DOWN}\cdot {U}_{kt}+\varepsilon, $$

(2)

where UP and DOWN are indicator variables whether unemployment increased or decreased, respectively, over the previous 3 months (UP is coded 1 when there is no change) and γ ^U and γ ^D represents the responsiveness of MJH to unemployment during periods of contraction and expansion (increasing and decreasing U), respectively.

The cross-sectional model with MSA fixed effects provides one method for examining how within-MSA MJH varies with MSA-specific changes in unemployment. An alternative approach is to estimate a longitudinal model, which has the added advantage of accounting for individual worker heterogeneity. Here, we regress individual changes in dual job status over 1 year among workers remaining in the same physical household residence. We estimate the following specification:

$$ \varDelta {M}_{ikt}=\beta '\;\varDelta {x}_{ikt}+\gamma '\;\varDelta {U}_{kt}+\varDelta \varepsilon, $$

(3)

where ∆ is the change operator (time period t now represents 1-year changes), and γ’ provides an estimate of multiple job change cyclicality, after accounting for individual heterogeneity. In the panel model, all labor market and many individual worker controls fall out since most worker attributes remain constant over a year. The dependent value takes on values of −1 (multiple job leavers), 0 (single and multiple job stayers), and +1 (multiple job joiners). This model can be estimated as shown above, or with U in levels included in addition to ∆U. Standard errors are clustered by MSA. In the model above, the reference group is “stayers”—those who remain in single or multiple jobs in both years and a treatment group that includes workers switching from either single to multiple or multiple to single jobs.

To investigate how occupational skill difference (referred to as “mismatch”) between primary and secondary jobs change with respect to the business cycle, we estimate regressions for the much smaller sample of workers who hold multiple jobs. The dependent variables, ∆skill and ∆working conditions, are equal to the differences in SK and WC, the O*NET indices of occupational skill requirements and working conditions, respectively, between workers’ primary job occupation and secondary job occupation. (i.e., the ∆ operator is not longitudinal, but the difference between jobs at a point in time).

$$ \varDelta \mathrm{skill}=\beta\;{x}_{ikt}+\theta\;{\mathrm{LM}}_{kt}+{\gamma}_{\mathrm{sk}}\;{U}_{kt}+\varepsilon, $$

(4)

$$ \varDelta \mathrm{working}\;\mathrm{conditions}=\beta\;{x}_{ikt}+\theta\;{\mathrm{LM}}_{kt}+{\gamma}_{\mathrm{wc}}\;{U}_{kt}+\varepsilon . $$

(5)

In equation (4), a positive (negative) γ _sk implies that the skill advantage seen for primary jobs will widen (narrow) as unemployment increases (decreases). In equation (5), a positive (negative) γ _wc implies working conditions in the primary relative to the secondary job will worsen (improve) as unemployment rises (higher WC values imply more demanding working conditions).

8 Evidence on MJH and the business cycle

Our analysis focuses on estimates of γ, which measure the response of MJH to differences in local labor market unemployment rates across markets and over time. Estimates are from regressions with very large sample sizes including a rich set of covariates. In Table 2, we present LPM estimates using three specifications. The “base” equation includes human capital and demographic characteristics, along with each worker’s wage on the primary job, plus fixed effects for hours, industry, occupation, region, metropolitan area size, year, and month. A “sub-base” specification omits region and city size fixed effects. In a third column, we add MSA fixed effects, which absorb the region and city size dummies.¹⁸ In all the results shown in the paper, standard errors are clustered on MSA to account for error correlation among workers within the same labor market. As evident in Table 2, estimates of γ are nearly identical (−0.19 and −0.21) in the sub-base and base specifications, respectively, but close to zero (−0.03) and insignificant following addition of MSA fixed effects.

Table 2 Unemployment effects on multiple job holding

Before discussing estimates of γ, we briefly summarize the effects of covariates on MJH. The full set of coefficients for the three specifications is shown in Appendix Table 10. Using our base equation, MJH increases with education, the MJH rate being 0.8 percentage points higher for high school graduates than dropouts, while bachelor’s degree and graduate degree workers have MJH rates 2.6 and 3.5 percentage points higher, respectively. Similarly, MJH rates increase moderately with age, before trailing off among workers in their 50s and 60s.¹⁹

Although raw MJH rates are slightly higher for women than for men, after accounting for covariates MJH rates are about 1.3 percentage points lower among women. Rates vary little with respect to race and ethnicity, although the signs on black, Asian, and Hispanic are negative (non-Hispanic whites are the reference group). Rates do not differ between natives and foreign-born citizens but are about a 0.5 percentage point lower for non-citizens. Workers never married and those married with spouse present have MJH rates roughly a percentage point lower than among those previously married but currently separated, divorced, or widowed. Those with preschool children in the household have a MJH rate about 0.5 percentage point lower than those either with children 6 or older or without children in the household. As expected, MJH decreases with respect to hours worked on the primary job (the omitted category of workers is those with variable hours on their primary job, who exhibit low rates of MJH). Union members have MJH rates about a half point higher than non-members; all else the same. Those with higher average hourly earnings in their primary jobs have a lower MJH rate, but the effect is small, a 10 % higher wage in the primary job being associated with a 0.04 percentage point lower MJH rate.

Our principal focus is on estimates of γ, the coefficients showing the response of MJH to differences in local labor market unemployment rates across MSAs and over time. Our large sample allows us to observe differences across groups not possible in previous studies. For the full sample, these estimates were shown in the top row of Table 2. In Table 3, we provide estimates of γ for alternative groups of workers and time periods from the “base” and “base + MSA FE” specifications. Clearly evident is that nearly all estimates of γ are negative, indicating that MJH is procyclical, expanding as unemployment decreases and receding as it increases. That said, the magnitudes of γ are not large. In our base equation for the full sample, the estimated γ is −0.207 and highly significant, indicating that each 1 percentage point increase in local area unemployment (an increase in U of 0.01) is associated with a 0.002 lower MJH rate (i.e., a fifth of 1 %). A one standard deviation increase in the unemployment rate (equal to 0.025) is associated with a 0.005 (half percentage point) lower rate of MJH. Relative to the mean multiple job rate of 4.7 % in our sample, an unemployment rate 1 percentage point higher in one labor market versus another is associated with a 4 % reduction in the MJH rate (i.e., −0.002/0.047 = −0.043). In short, the across-MSA cyclical association between unemployment and MJH, conditional on worker, job, and broad locational controls is systematically negative and statistically significant but modest in size.

Table 3 Unemployment effects on multiple job holding, by worker group

When we add MSA fixed effects, which absorb region and city size effects, estimates of γ fall sharply, are insignificant, and close to zero. For the full sample, the estimate of γ with MSA fixed effects is −0.030, about one seventh as large as the estimate from our base specification. The point estimate implies that a 1 percentage point increase in unemployment (an increase of 0.01 in U) is associated with near-zero reduction in MJH (a 0.0003 reduction in the 0.047 dual job rate across this sample). In short, within US labor markets, the response of MJH to within-market changes in unemployment is largely acyclic.

Table 3 enables us to examine variation in multiple job responsiveness to unemployment (i.e., estimates of γ) across different groups of workers. Given that nearly all the estimates including MSA fixed effects are close to zero, our discussion focuses on results without as well as with MSA fixed effects. Overall, estimates of γ are highly similar across groups and time periods. As discussed earlier, we expect differences between men and women due to household specialization to show up most strongly for married rather than single workers.²⁰ Income effects (leading to a less negative or positive γ) should be more substantive for married workers (women or men) than for single workers. These qualitative results are seen in Table 3. Moreover, we do not see this same relationship when we compare all men and women or single men and women. Among all groups of workers shown in Table 3, married women have estimates of γ that are closest to zero (least negative) and, in the specification with MSA fixed effects, a small positive coefficient. Given the absence of market clearing in a recession, it is not possible to disentangle net differences between labor supply income and substitution effects from those of demand effects (vacancies) that vary with business conditions. The overall net effect, however, is a largely acyclic pattern of MJH among married workers. We also examine multiple job response among foreign-born and native workers and find nearly identical estimates of γ in the base equation.

Differences in MJH with respect to business conditions among hourly versus salaried workers in their primary job have not been previously examined. Hourly workers display a somewhat more cyclical pattern than do salaried workers, −0.234 versus −0.155 in the base specification. To the extent that hourly workers have more variable earnings due to variation in regular and overtime hours, they should have stronger income effects and thus display less cyclical behavior than do salaried workers. But we observe the opposite. The implication is that observed multiple job outcomes are driven strongly by labor demand (vacancies), with second job opportunities for hourly primary workers being cyclical and difficult to find during downturns.

An additional question to address is whether and how the relationship between MJH and local labor market unemployment changed during the Great Recession, a period in which the rate of nationwide MJH changed little, but in which there was substantial variation across labor markets in unemployment. While we found a value of γ = −0.207 in our base equation over the entire 1998–2013 period, restricting the sample to 2008–2010 (the Recession began officially in December 2007), we obtain a similar but slightly more negative γ estimate of −0.231 during the Great Recession. Splitting our entire 1998–2013 in half, we find that γ, if anything, became slightly less cyclical over time, but differences are small and insignificant. Although the level of primary and secondary job holding declined in the Great Recession, the aggregate MJH rate declined only modestly, not markedly different from declines seen prior to 2008 and after 2010. Although the Great Recession produced large household income effects and no doubt increased the search for second jobs, few such jobs were available. In short, MJH, on net, appears to have done little to either mitigate or exacerbate income losses during the Great Recession.

9 Further evidence and robustness checks

Below we summarize several extensions of our principal analysis. In sections 9.1 and 9.2, we examine MJH cyclicality using two additional measures of the business cycle, MSA-specific employment growth and growth in job ads (vacancies). Section 9.3 addresses the question of whether the response of MJH with respect to the business cycle is symmetric with respect to increases versus decreases in unemployment. Sections 9.4 and 9.5 provide longitudinal analysis based on short CPS panels with two observations on each worker, 1 year apart, and in the same residence (and hence MSA). First addressed is the question whether given workers change their MJH status with respect to changes in unemployment within a given labor market. Next addressed is whether husbands and wives change MJH status during the past year in response to labor market changes by their spouse.

9.1 Employment growth as a business cycle measure

As a robustness check, the log of employment growth is used as an alternative business cycle measure. We use employment data from the Quarterly Census of Employment and Wages (QCEW). The QCEW provides monthly data on employment at the county level, which in turn can be aggregated up to the MSA level. In order to reduce measurement error in local employment counts, we use as our measure of employment growth the average monthly log employment growth in the 3 months surrounding the CPS reference week. For example, workers surveyed in the CPS in mid-March would be matched to the average log difference in QCEW employment for Jan/Feb, Feb/Mar, and Mar/Apr.

MJH employment growth results are shown in Table 4. The format is identical to that seen previously in Table 3 using the unemployment rate as the business cycle measure.²¹ Three results are evident. First, all coefficients are positive, consistent with the procyclical MJH found previously using the unemployment rate as the business cycle measure. Second, the magnitudes of the point estimates both with and without MSA fixed effects are minuscule. For example the roughly 0.10 coefficient estimates seen for the full sample in both the base and fixed effects specifications imply that employment growth of 1 % (0.01), would be associated with a 0.001 (one tenth of 1 %) increase in MJH. The close-to-zero effect of employment changes on MJH aligns closely with our Table 3 estimates using the unemployment rate measure with MSA fixed effects. Third, the employment change measure used in Table 4 has little correlation with the unemployment rate measure in Table 3. Rappaport (2012) likewise finds little correlation between metro area employment growth and unemployment rates. Each is a legitimate measure of labor market tightness, but these are conceptually different measures. The unemployment rate is perhaps a more direct measure of cyclicality, reflecting the ease or difficulty in finding a job. The employment growth measure largely reflects differences across urban areas in economic growth rates.

Table 4 Employment growth effects on multiple job holding, by worker group

The robust negative correlation between MJH and unemployment across labor markets, absent MSA fixed effects, clearly reflects systematic differences across labor markets in MJH that are correlated with relatively fixed differences in unemployment levels. Labor economists have long been aware of systematic long-run differences in unemployment across markets (e.g., Hall 1970 and, more recently, Rappaport 2012). To illustrate the difference in labor market “fixity” between our alternative business cycle measures, we simply regress each of their monthly rates on MSA fixed effects for our 1998–2013 sample period. In contrast to an R ² of 0.01 for the employment growth regression, we obtain an R ² of 0.43 for the unemployment regression. Adding year and month fixed effects to each increases their respective R ² values to 0.26 and 0.83. The conclusion that unemployment rates have greater fixity across markets than does employment growth is robust to use of weights, use of 3-month versus 1-month unemployment and employment measures, and to expanding the analysis to include small MSAs not identified in the CPS. Although economists are well aware of the systematic long-run differences in area unemployment, there has been little notice of systematic differences across states (or MSAs) in MJH over time. An exception is Partridge (2002), plus a recent analysis by Hirsch et al. (2016) that focuses on explanations for local labor market fixity in MJH.

Although we regard the unemployment rate and employment growth as alternative business cycle measures, the low correlation between the two allows us to jointly include both measures in MJH regressions and examine their separate effects. The top three panels of Table 5 show coefficient estimates on the unemployment rate and employment growth from MJH regressions for our base and fixed effects (FE) specifications. Shown are the coefficients when both measures are included, as well as those seen previously (in Tables 3 and 4) when the measures were included separately. Including both measures reduces the absolute value of the coefficients, but the reduction is small. Using the base specification with both measures included, we obtain coefficient estimates of −0.205 and 0.081 on the unemployment rate and employment growth, respectively, compared to −0.207 and 0.103 seen previously when entered separately. With MSA FE included, we obtain estimates of −0.024 and 0.089 on the unemployment rate and log employment growth, compared to the separate −0.030 and 0.091 estimates. Standard errors are likewise similar when included separately and jointly. These proxies for labor market tightness and the business cycle can be treated as distinct measures.

Table 5 Effects of unemployment, employment growth, and job ads on multiple job holding

9.2 Labor market job ads (vacancies) as a demand measure

Whereas the unemployment and employment growth measures reflect both labor demand and supply, a measure of job advertisements (vacancies) is arguably a more direct measure of labor demand, albeit one influenced by labor supply (worker availability). In this section, we examine the relationship between MJH and job ads, conditional on unemployment or employment growth. The Help Wanted OnLine (HWOL) job ad data are collected by The Conference Board. We use a measure of monthly job ads collected at the MSA level for 52 large MSAs beginning in late 2005.²² The MSA job ad data are highly correlated with our MSA employment measure. Like the employment measure, the job ad data are not highly correlated with the unemployment rate across labor markets.

In the bottom two panels of Table 5, we examine the relationship between individual worker MJH and the MSA growth in job ads (log changes in 3-month averages). The sample size is roughly a third of our previous analysis given the limited number of years and MSAs for which the HWOL data are available. That said, the coefficients on the unemployment rate and our control variables are highly similar to those seen previously. When we include both the job ad growth rate and the unemployment rate, we obtain a small, significant (at the 0.05 level) coefficient of 0.02 on job ad growth in both the sub-base and base specifications. This implies that 0.10 log growth in help wanted ads is associated with two tenths of a 1 percentage point increase in the MJH rate. When we add MSA FE, the coefficient is about half as large and no longer significant. In the bottom panel of Table 5, we jointly include employment growth and job ad growth, two highly correlated measures of labor demand. All coefficients are positive but are tiny in magnitude and insignificant. These results reinforce our prior conclusion that there exists little cyclicality in MJH.

9.3 Symmetric response to unemployment

An additional check on prior results is to ask whether MJH responds symmetrically to increases and decreases in unemployment. To examine this question, we estimate separate coefficients on unemployment for periods of contraction and expansion, with unemployment interacted with UP and DOWN indicator variables based on whether unemployment increased or decreased in the previous 3 months (equation (2)). Results are shown in Table 6. Using our base equation, coefficients are highly similar to the −0.207 (0.030) estimate seen previously in Table 2, being −0.206 (0.030) during contractions and −0.208 (0.029) during expansions. With MSA FE included, coefficients are nearly identical to the insignificant −0.030 (0.023) estimate seen previously in Table 2, being −0.030 (0.023) during contractions and −0.029 (0.023) during expansions. As an additional check, we provide these estimates using 1-month rather than 3-month measures of both the unemployment rate variable and the indicator variable for UP and DOWN. The absolute value of the coefficients is slightly lower using the 1-month versus 3-month measure, presumably due to measurement error in the former. Coefficients are again nearly equivalent during expansions and contractions.

Table 6 Asymmetric response of multiple job holding with respect to unemployment

Summarizing our results, we find that MJH with respect to the two business cycle measures is weakly cyclical, but sufficiently close to zero so as to characterize the relationship as acyclic. Moreover, we find no evidence for asymmetry in MJH responses over the business cycle.

9.4 Longitudinal evidence on MJH

As a check on the cross section results, we estimate MJH regressions using short 2-year CPS panels of workers surveyed in the same residence and month in consecutive years. The main advantage of the panel evidence is that it accounts for worker heterogeneity that may be correlated with MJH and unemployment. Rather than comparing a group of multiple job holders with a group of single job holders, conditioning on measured attributes, we compare each worker to herself 1 year apart, with some workers having switched between single and multiple job holding.

Our estimation sample is for the worker/year pairs 1998/1999 through 2012/2013. Results are shown in Table 7. Estimating longitudinal equation (3) shown previously, we obtain an estimate of γ’ (the coefficient on ∆U) of −0.037 (0.027). The sample size is 570,939. This coefficient is less than a fifth of the size of the −0.207 cross-sectional estimate of γ in our base specification, being close instead to the −0.030 estimate from the MSA FE specification in Table 2. Adding a control for the unemployment level U, the coefficient on ∆U barely changes. When we add dummies for year and month, thus allowing MJH transitions to differ among months and over time, estimates of γ’ turn positive, being effectively zero and not close to significance. In short, the longitudinal evidence reinforces our previous conclusion based on cross-sectional analysis with MSA FE. There is little or no net relationship between MJH and unemployment within labor markets.

Table 7 Longitudinal analysis of multiple job holding

9.5 Household shocks and MJH

As discussed earlier, it is difficult to distinguish between MJH labor supply and labor demand responses over the business cycle. As stated in the introduction, there is a widespread belief that MJH increases during recessions owing to negative income shocks. Yet the clear result from our analysis is that MJH is weakly procyclic or acyclic. No doubt there are many workers who wish to work at multiple jobs during a downturn but not all will find such jobs. We cannot cleanly measure the separate effects of demand and supply forces, but in this section, we provide evidence in Table 8 showing that household income shocks influence MJH behavior as expected.

Table 8 Panel analysis of married workers’ MJH response to spouses’ labor market transitions

In order to examine household income shocks, we return to our 1998/1999 to 2012/2013 longitudinal sample used in the prior section. We examine the impact of husbands’ (wives’) labor force transitions during the past year on wives’ (husbands’) changes in MJH status. Hence, our sample is restricted to persons married in both periods who are designated as either the head of household or married to the household head. Results are estimated separately for wives and husbands with parallel structure. Columns 1–3 in Table 8 measure wives’ responses to husbands’ transitions; in a parallel structure, columns 4–6 show husbands’ responses to wives’ transitions. These include the four possible transitions between employment (E) and either unemployment (U) or not in the labor force (N). We also control for a spouse not employed in either period (NN), with the omitted reference group being a spouse employed both periods (EE). The first specification includes differences of the non-fixed basic controls seen previously (with the exception of hours worked on the primary job), plus the unemployment rate in changes and level. The second adds year pair and month dummies. The third adds a variable measuring the log change in spousal real weekly earnings, which should absorb some of the effects from labor market transitions seen in the prior specifications.

As expected, transitions into MJH are increased (by 0.01) among wives whose husbands have moved from employment to unemployment. This is a substantive effect given the mean MJH rate of 0.042 for this sample of married women. The coefficient effectively goes to zero when one controls for husbands’ changes in weekly earnings (column 3). Movement into MJH among wives is slightly higher among those with husbands not employed in either period.

Columns 4–6 of Table 8 provide results for the parallel analysis of husbands’ MJH transitions. We observe no evidence that husbands increase MJH in response to their wives moving from employment to unemployment (once earnings change is controlled for in column 6, the EU coefficient in column 6 turns negative). That said, we do observe evidence for an income effect in column 6, where husbands’ MJH behavior is inversely related to wives’ changes in weekly earnings. An interesting pattern seen in husbands’ behavior is that they appear to coordinate leisure and/or home production time with their wives. Husbands are more likely to move out of dual job holding when their wives transition from employment to not in the labor force (E to N) and, more substantively, move into dual job holding when wives move into employment (N to E). The same qualitative pattern was seen in wives’ behavior with respect to husbands’ transitions, but the magnitudes were much smaller and largely insignificant.

Although suggestive, these results potentially reflect endogeneity due to reverse causality or omitted variables. For example, imagine a struggling couple in which the husband works at two jobs and the wife at one. If the husband obtains a high-paying primary job, he may move out of MJH and his wife may drop out of the labor force. This would produce reverse causality effects in the husband response equation. Alternatively, there could be a health/caregiving shock to the family, leading one spouse to quit a second job and the other spouse to exit the labor force.

Given that male unemployment is likely to be exogenous, we do find believable the effects of husband E to U transitions or change in earnings on their wives’ MJH transitions. These results suggest a MJH labor supply increase due to a negative income shock within households, consistent with theory. It is worth noting, however, that such responses do not produce large macro outcomes. A small proportion of individuals are unemployed; a small proportion of persons hold multiple jobs. The number of households with one spouse unemployed and the other a multiple job holder is minuscule.

10 The business cycle and occupational mismatch between primary and secondary jobs

Our final analysis examines how job quality differences between primary and secondary jobs vary with the business cycle. As previously observed in Table 1, on average, multiple job holders’ second jobs have a lower O*NET occupational skill index (SK) value and a higher (more demanding) physical working conditions (WC) value, signaling lower “quality” on secondary than on primary jobs. Given that the skill index is highly correlated with pay, while working conditions are not, these results suggest that on average workers’ second jobs pay less than do their first jobs (our guesstimate is roughly 6 % less). Not examined in previous literature is whether job quality differences vary over the business cycle. Based on the sample of multiple job holders, we estimate regressions in which the dependent variable is the difference in either SK or in WC between the first and second jobs (equations (4) and (5)). Our focus is on how these differences in job quality vary with the business cycle, with our expectation being that during recessions the job quality gap might widen due to more limited options among workers.²³

As seen in Table 9, the qualitative effects of unemployment on job quality differences are contrary to our expectations, but very small in magnitude. The gap in occupational job quality between the primary and secondary job narrows slightly during recessions and widens in expansions. The coefficient on the unemployment rate is −1.62 (significant at the 0.01 level) for the ∆SK base equation. It declines to −0.53 and is insignificant when MSA FE are added. These estimated effects are tiny. The raw mean gap in occupational SK between first and second jobs is 0.288 (see Table 1), more than a quarter of a standard deviation difference. Coefficient effects of −1.62 or −0.53 multiplied by a 0.01 (1 percentage point) increase in unemployment are trivial relative to the standard deviation in SK (by construction 1.0) or compared to the 0.3 mean difference between SK1 and SK2.

Table 9 Unemployment effect on primary minus secondary job differences in skills and working conditions

Turning to the ∆WC equations, we obtain a significant (at the 0.05 level) coefficient on the unemployment rate of 0.75 for the base equation. Adding MSA FE, the coefficient is insignificant and effectively 0, at 0.23. The mean difference in WC between jobs (Table 1) is a small 0.054 points, with slightly more demanding working conditions in the second job. The positive coefficient on the unemployment rate implies that as unemployment increases, the small advantage of less demanding working conditions in the primary relative to the second job narrows slightly. The magnitude of these effects is trivial. For both the skill and working condition regressions with MSA FE, coefficients are far from statistical and substantive significance.

In short, there are modest occupational job quality differences between dual workers’ first and second jobs, with the primary job having SK and WC index values indicating moderately higher skills and slightly less demanding working conditions. These gaps in job quality are largely invariant to differences in unemployment across markets or changes within markets over time.

11 Conclusions

MJH has the potential to exacerbate or mitigate employment and income changes over the business cycle, depending on whether the relationship is cyclical or countercyclical. Theory is helpful, but inconclusive; substitution and income effects in labor supply for multiple jobs typically cut in opposite directions. And whatever the preferred labor supply responses for multiple jobs over the cycle, such preferences can be demand constrained. Even if labor supply were countercyclical, the availability of second jobs is likely to be cyclical absent highly flexible wages. Empirical evidence is needed to determine how MJH varies over the cycle. Our CPS data set, which includes large samples of workers with MJH information over many years, provides precise estimates of these net effects.

As stated in the introduction, anecdotal evidence from news reports suggests that MJH is countercyclical. Our evidence clearly indicates that to the extent that MJH responds to unemployment, the net response is procyclical, but tiny in magnitude. No significant relationship exists between MJH and business cycle measures once one accounts for MSA FE. Longitudinal analysis confirms that there is minimal response of MJH in response to unemployment or employment growth changes within markets over time. In contrast to prior US studies, our CPS data set produces relatively precise estimates, albeit ones close to zero.

Because there is little variation in MJH with respect to market conditions within MSAs over time, it follows that national employment figures are affected little by business cycle changes in MJH. Individual households vary their job holding over time in ways that benefit them financially and enhance well-being. Some of that variation is cyclical and some countercyclical. But in the aggregate, MJH is largely acyclic and plays a (perhaps surprisingly) small role in mitigating earnings volatility over the business cycle.

Appendix

Table 10 Unemployment effects on multiple job holding, full coefficient results