Are Unemployment Numbers Bogus?

June 04, 2009


Employment and unemployment numbers have a prominent position in the financial news, but many investors and commentators take an over simplified view of what they mean. All too often, a discussion of unemployment announcements is dismissed with the statement: "... is a lagging indicator.”
In two separate articles, I have discussed some ways that employment data can be used to determine where we are in the economic cycle and how to use the data to guide investment strategy. Many have criticized the published data as being unreliable because of modeling corrections and adjustments. In this article we will try to ask hard questions about the reliability of the data published.

What Are the Questions?

There has been a lot of chatter over the past few months about how to interpret employment reports and whether the unemployment numbers reported weekly and monthly by the U.S. Dept. of Labor are accurate, or even misleading. These are some of the questions:

• How many different surveys and reports are there?
• Are the weekly initial unemployment claims missing some people entering unemployment?
• Are the seasonal adjustments valid? Should the historical seasonal adjustment be applied when conditions deviate significantly from historical norms?
• How can it be that the increase in unemployment reported each month is a small fraction of the total of all weekly initial unemployment claims during that month?
• Should all people entering the workforce be added to the total of the employed (removed from the rolls of the unemployed) in the same way? Should temp employment be counted the same way as a permanent hire?
• Is the total number of employed people understating the number of people actually working? Are self-employed, part-time employed and people “off the books” counted?

A good summary of measurement and estimation methods and issues for monthly employment reports can be found here and here. Specific topics will be discussed in this article.

Why Are There Two Monthly Surveys?

The two surveys, the household survey and the establishment survey (also called the payroll survey), have two different objectives. The household survey objectives include estimating the employment status of the entire population and making some demographic assessments. The establishment survey objectives include assessing the industry and government group distributions of employment and making connections to unemployment claims data. If someone is working two jobs that are in the industry and government groups covered, that person could be counted twice if both jobs are surveyed. Multiple jobs are counted as one employed person in the household survey. More information about the two surveys can be obtained here and here.

In addition to the two monthly surveys, initial unemployment claims and continuing insured unemployed are reported weekly. In this article, both the monthly household employment survey and the weekly initial unemployment claims reports will be analyzed.

What Are Weekly Initial Unemployment Claims Missing?

Weekly initial unemployment claims do not count all people losing employment. Only those eligible for unemployment insurance benefits are counted. While some states cover all part time employment above some minimum number of hours, many states cover only full time employment. Part timers (in many but not all states), some temporary workers, contract workers, self-employed and off-the-books workers do not get picked up in this number when they lose their employment.

I do not have an estimate of how many people fall into these uncounted categories. It would not be surprising if it was several 10’s of thousands. There is no reason to suspect that these categories would not suffer loss of employment at a rate at least as high as covered employees. There are definitely significant numbers of people losing employment each week who are not counted. The official Insured Unemployed Rate as of May 2 is 4.8% (seasonally adjusted), which is a little more than half of the official total unemployment rate for April of 8.9%. The implication is that there may be a number of uninsured job losses each week, up to almost as many as the insured number.

Of course, there are people leaving the insured unemployed ranks each week, some who find jobs and some whose benefits expired. In the week of April 25, that number was 478,000. Since, as we shall see later in the article, there was an increase in the ranks of the employed of 120,000 for the entire month of April (the ranks of the unemployed increased by much more), the number of 478,000 is mostly composed of still unemployed people whose benefits ran out, probably more than 400,000.

What About Seasonal Adjustments?

According to the U.S. Dept. of Labor Bureau of Labor Statistics:

Over the course of a year, the size of the nation's labor force and the levels of employment and unemployment undergo sharp fluctuations due to such seasonal events as changes in weather, reduced or expanded production, harvests, major holidays, and the opening and closing of schools.
Because these seasonal events follow a more or less regular pattern each year, their influence on statistical trends can be eliminated by adjusting the statistics from month to month.

While adjusting for past seasonal variables may be quite reproducible from year to year in an economy operating on an even keel, they may (or may not) be applicable in the same way in times of economic upheaval. This is an area that must be viewed as introducing possible bias when conditions in a certain month of the year are subjected to a much different economic environment than the same month in prior years. This uncertainty applies to both weekly initial unemployment claims and total unemployment estimates.

Monthly Employment Gains (Losses)

Let’s look at the recent report for April from the Dept. of Labor, released May 8.



One thing that stands out is the demographic distribution of unemployment. Teens and minorities are disproportionally unemployed. Another thing is the unemployment rate for men is 32% higher than women (9.4% vs. 7.1%). Only education, health services and government showed an increase in employment in April.

The biggest question about these numbers is the disparity between the increase in unemployment each month and the total of weekly initial unemployment claims. The 563,000 reported increase in unemployment for April is much smaller than 2,653,000 the total of weekly initial unemployment claims for the four April reporting dates.
Note: 2.6 million covers claims from March 29 through April 25, the four reporting weeks most closely aligned with the month of April.

Below is the claims data for 2009 through May 2. (From The U.S. Dept. of Labor.)



The more recent data is given in the latest press release Thursday (May 28)



The numbers that should be reconciled in calculating the change in employment from one month to the next are shown in the following table, with Dept. of Labor published initial unemployment claims data and estimates (household survey data), where available.


The total of X – Y = 2,533,000 net new jobs created. In other words, 2.65 million put in initial unemployment claims, plus another Y uninsured lost their jobs in April. This was offset by 2.53 million X who found employment. The Dept. of Labor does not track these millions of individual moves. They simply estimate the number of employed (120,000 increase in April) and the number of unemployed (563,000 increase in April) for the month. These totals for the month are the results of millions of changes occurring in and out of employment.

Estimated Measurement Errors and Data Uncertainty

The Dept. of Labor makes the following statement regarding uncertainty in the reported numbers:

The establishment survey employment series has a smaller margin of error on the measurement of month-to-month change than the household survey because of its much larger sample size. An over-the-month employment change of 107,000 is statistically significant in the establishment survey, while the threshold for a statistically significant change in the household survey is about 400,000. However, the household survey has a more expansive scope than the establishment survey because it includes the self-employed, unpaid family workers, agricultural workers, and private household workers, who are excluded by the establishment survey.

The uncertainty referred to above is at the 90% confidence interval. This varies considerably from month to month. For example, in another section of Dept. of Labor literature, the following statement appears:

When a sample rather than the entire population is surveyed, there is a chance that the sample estimates may differ from the "true" population values they represent. The exact difference, or sampling error, varies depending on the particular sample selected, and this variability is measured by the standard error of the estimate. There is about a 90-percent chance, or level of confidence, that an estimate based on a sample will differ by no more than 1.6 standard errors from the "true" population value because of sampling error. BLS analyses are generally conducted at the 90-percent level of confidence. For example, the confidence interval for the monthly change in total employment from the household survey is on the order of plus or minus 430,000.

In the May 8 report covering April, the uncertainty is given as +/- 436,000 (table on page 4).

The household survey covers approximately 60,000 households each month. The following two table estimates on how this relates to the 18-65 age population.

We can use the numbers above to estimate how each response error reflects on the entire population age 18-65. The each individual covered by the survey represents approximately 2,400 people (235,000,000 divided by 97,800). Thus, an uncertainty of +/- 400,000 corresponds to approximately 167 employment status classification errors in each survey. This has to cover:

• Misunderstood questions
• Incorrect classification of responses
• Deliberately incorrect responses
• Processing errors after the interviews

If there are 50 survey interviewers (an arbitrarily assumed number - the Bureau of Labor Statistics says that 2000 people are involved in carrying out the household survey), that would allow about 24 minutes per survey for a 24 workday month (8 hours a day). This means that only 3.33 errors would occur per month for each interviewer. To verify this low number of errors, a study could be run to exactly duplicate surveys for several months, using different polling interviewers of the same respondents (and, in parallel, different respondents) and analyzing the results at each stage of the process. Has this ever been done?

It is certainly reasonable to question whether or not the sampling error is possibly understated. The larger the sampling error is each month, the more months must be accumulated in order to average out sampling errors and establish a trend. Perhaps we should be paying attention to the 2-, 3- or 4-month average of employment and unemployment figures.

The Birth/Death Adjustment

The Dept. of Labor makes a monthly model based estimate of jobs created by the birth of new businesses and jobs lost through the death of businesses.

The monthly establishment survey estimates include an adjustment to account for the net employment change generated by business births and deaths. The adjustment comes from an econometric model that forecasts the monthly net jobs impact of business births and deaths based on the actual past values of the net impact that can be observed with a lag from the Quarterly Census of Employment and Wages. The establishment survey uses modeling rather than sampling for this purpose because the survey is not immediately able to bring new businesses into the sample. There is an unavoidable lag between the birth of a new firm and its appearance on the sampling frame and availability for selection. BLS adds new businesses to the survey twice a year.

In April, the Dept. of Labor Bureau of Labor Statistics estimated that 226,000 jobs were created due the birth of new businesses. This number is net of losses due to business deaths. If this would change the numbers discussed previously (Monthly Employment Gains section), it would constitute a small fraction of the implied total of people finding new jobs. However, the previous discussion was based on the household survey, which it seems should include the effects of business births and deaths without adjustment, because the survey is not reaching people through businesses, it is surveying people directly (household survey). I have found the following questions and answers:

Q: Can I subtract the birth/death adjustment from the seasonally adjusted over-the-month change to determine what it is adding to employment?
A: No. Birth/death factors are a component of the not seasonally adjusted estimate and therefore are not directly comparable to the seasonally adjusted monthly changes. Instead, the birth/death factor should be assessed in the context of its effect on the not seasonally adjusted estimate.
Q: Can BLS provide an estimate of the contribution of the birth/death adjustment to the seasonally adjusted monthly payroll change?
A: BLS does not calculate an estimate of the seasonally adjusted contribution of the birth/death model. The sample, the imputation of business births using deaths, and the net birth/death model are all necessary components for obtaining an accurate total employment estimate. The components are not seasonally adjusted separately because they do not have any particular economic meaning in and of themselves.

This does not make clear whether the birth/death adjustment applies to the total employment numbers (from the household survey) or not. Reviewing three other publications, here, here and here, has not provided any further clarification.

Are People Falling Through The Cracks?

The Dept. of Labor maintains that their sampling techniques are getting good estimates for much of the labor force. The following table was constructed from April, 2009 seasonally adjusted data here and here.

This estimate of the total work age population seems a little high. Using the latest age distribution data from the U.S. Census Bureau (here) from July, 2007 and multiplying by 1.01 to correct for population growth since, I calculate the U.S. population from age 15 through 79 to be about 231,000,000.

It doesn’t look like the Dept. of Labor is missing people. If anything, they may have over counted slightly.

Even if the total working age population is about right, what about the 80 million in the age group but not in the work force. This presumably includes all not working because of long term illness and disability, and those not working by choice. The Dept. of Labor makes the following definitions:

The civilian labor force is the sum of employed and unemployed persons.

Those not classified as employed or unemployed are not in the labor force.
The unemployment rate is the number unemployed as a percent of the labor force.
The labor force participation rate is the labor force as a percent of the population, and the employment-population ratio is the employed as a percent of the population.
People are classified as employed if they did any work at all as paid employees during the reference week; worked in their own business, profession, or on their own farm; or worked without pay at least 15 hours in a family business or farm. People are also counted as employed if they were temporarily absent from their jobs because of illness, bad weather, vacation, labor-management disputes, or personal reasons.
People are classified as unemployed if they meet all of the following criteria:

• They had no employment during the reference week; and
• They were available for work at that time; and
• They made specific efforts to find employment sometime during the 4-week period ending with the reference week.
• Persons laid off from a job and expecting recall need not be looking for work to be counted as unemployed.

Note: The preceding statement separated into bullets by the author for readability.

So the people “not in the labor force” includes all people without employment (except for those with an anticipated future return from a layoff) who have not made a specific effort to find employment within the past four weeks. This is presumably a large number of people, probably larger during times of economic contraction. This group includes “marginally attached workers”, which includes “discouraged workers”. The April report stated:

About 2.1 million persons (not seasonally adjusted) were marginally attached to the labor force in April, 675,000 more than a year earlier. These individuals wanted and were available for work and had looked for a job sometime in the prior 12 months. They were not counted as unemployed because they had not searched for work in the 4 weeks preceding the survey. Among the marginally attached, there were 740,000 discouraged workers in April, up by 328,000 from a year earlier. Discouraged workers are persons not currently looking for work because they believe no jobs are available for them. The other 1.4 million persons marginally attached to the labor force in April had not searched for work in the 4 weeks preceding the survey for reasons such as school attendance or family responsibilities.

Table A-13, attached to the April Dept. of Labor report, indicates that only 5,935,000 (including the 2.1 million marginally attached) persons in the labor force age group but not in the labor force actually desired work. This was an increase of 121,000 from March. Thus, approximately 75 million, according to the Dept. of Labor, do not want to work. Some may be full time students, some may be retired, some may suffer from disability and some may be economically dependent on a spouse or significant other. Some may be working in the underground economy, “off the books”. Among these 75 million, it is possible that people are falling through the cracks in significant numbers. There may be many people in this group who want to work but are not being included in the labor force.

Alternative Unemployment Measurements

The Dept. of Labor reports alternative measurements of unemployment, including U6, an estimate of unemployment and under employment. It is defined as:

Total unemployed, plus all marginally attached workers, plus total employed part time for economic reasons, as a percent of the civilian labor force plus all marginally attached workers.

In April, this was 15.8%. Many think this is a better indication of employment conditions than is the official unemployment rate.

Others have speculated about how many of the people not in the labor force are classified that way, but shouldn't be. These are people who have stopped looking because of acceptance of permanent discouragement, but still desire to work. Include these people and unemployment estimates range from the high teens to the mid-twenties (percent). For an example, see Shadow Stats.

Summary

The official unemployment numbers are not bogus, but they are subject to debate about definitions, large sampling uncertainty in the headline number and argument regarding interpretation. The biggest questions arise from the discrepancy between how many initial unemployment claims are filed each month and the official numbers of increased unemployment and employment for the same month. In April, this analyst can not account for approximately 2.5 million people who filed unemployment claims and do not appear to show up in the official employment/unemployment estimates. This is apparently the approximate number of people who returned to employment during the month.

Because of large sampling error uncertainty each month, it would be better to use multi-month averages to track the numbers of employed, unemployed and people not in the work force. Although this would delay the interpretation time for the information, the uncertainty would be reduced.

It is possible that there is enough “underground” employment (off the books) that the health of the economy, as indicated by employment numbers, is not described as well as it could be. This may be particularly true for people moonlighting while receiving unemployment benefits, so the problem is greater in times of high unemployment. From this perspective, unemployment numbers may be over estimated.

Finally, different unemployment data reports have different roles to play in understanding the business cycle. Weekly initial unemployment claims can be used as an alert for the potential of the start of a recession and as a leading to concurrent indicator for the end of a recession. The peak in unemployment is always a lagging indicator regarding the end of recession, but still has value in closing the last page of the recession book. As pointed out here, a peak in insured unemployment (published with the weekly initial unemployment claims report) is a much timelier indicator that a recession is ending. Also, it was determined here that employment information useful in making strategic investment timing decisions coming out of recessions does not include any of the statistics we have found reason to question in this article.


By John Lounsbury, http://piedmonthudson.wordpress.com
John Lounsbury -- Seeking Alpha