| |
|
Monday, March 12, 2001
|
|
|
Statistical Adjustment of the 2000 Census: What We Know
|
|
A Briefing Paper presented to Secretary of Commerce Don Evans by the Congressionally-appointed Members of the US Census Monitoring Board on March 1, 2001.
B>The census has TWO principal requirements:
- Count EVERY person living in America.
- Count EVERY person at the CORRECT ADDRESS — the ONLY guarantee for the fair distribution of political representation.
It is important to recognize the success of Census 2000 — the differential undercount was substantially reduced and the census number 281,421,906 is closer to the truth than any previous census has ever been. The Bureau’s strides in local hiring, reaching out to local partners and advertising, preliminary analysis indicates may have contributed to this success. However, the discussion regarding Census 2000 has not ended with this success.
Reasonable people may disagree whether or not statistical adjustment can "improve" the 2000 census enumeration. Statistical adjustment is more complicated than most people believe and the members of the ESCAP committee, a group of dedicated career professionals at the Census Bureau, have proven that scientific experiments do not always turn out as expected. After four months of concerted, almost seven-day-a-week-analysis, the highly respected statisticians recommended against the use of adjusted data. Nevertheless, it is still worthwhile for those outside of the ESCAP process to examine adjustment methodology and to review what most people who have carefully studied adjustment will agree on.
1. Statistical adjustment is NOT the Same Thing as Sampling.
This is NOT a debate about the scientific validity of sampling.
Originally the Census Bureau planned to count 90 percent of the population and "sample" the rest. A Supreme Court decision forced the Bureau to conduct a full (100 percent) enumeration. The issue for 2000 was whether or not to adjust that count using the results of a survey that the Bureau calls the Accuracy and Coverage Evaluation (A.C.E.).
Even though the census was not adjusted for the purposes of political redistricting, the A.C.E. will be used as a means to evaluate the strengths and weaknesses of the census data. No one objects to evaluating the census in order to develop strategies to improve the census and to identify the undercount. No one objects to the use of modern scientific methods as an evaluation tool; in fact, Congress supported modern scientific methods and appropriated every dollar needed to implement the A.C.E.
But, what is not known is whether using this analysis to reformulate the count for every census block across the nation is within the capacity of the A.C.E.`s methodology. Indeed, the career professionals at the Census Bureau have, since the formation of the ESCAP committee in late summer 2000, tried to answer that question and at this time have recommended the use of unadjusted data for the purposes of political redistricting.
2. A Complex Mathematical Process
Statistical adjustment is a complex mathematical process that begins with a comparison of the results from one of the largest sample surveys in history, 314,000 housing units, to the results of the census. The Census Bureau determines whether there are overcounts or undercounts based on this comparison. The Bureau will create adjustment factors based these comparisons for 448 "post-strata specified subgroups" within the population such as White Male Non-Owners (or renters) in the Northeast, age 18-29; Black Female Owners, age 50+; or Hispanic Female Owners, 30-49.
Consider for instance the following example, there are 800 census records for White Male renters in large metropolitan areas in the sample and that the A.C.E. has records for 808 White Male renters in large metropolitan areas at the same or comparable addresses, therefore the Census Bureau will determine that there is an approximate one percent undercount for this particular subgroup. This one percent undercount means that an adjustment factor of 1.01 will be applied to the population of approximately 400,000 people and approximately 4,000 will be added through adjustment in a somewhat random fashion. In other words, based on a sample of 800 people the Bureau will make adjustments to 400,000 people.
Moreover, all surveys have a margin of error. The Bureau has not indicated what the margin of error and corresponding bias for each geographic area will be and this ought to be known before any decisions about adjustment are made.
3. A Big Assumption: Comparing the Results of the Survey to the Census.
You have to make a lot of untested assumptions to statistically adjust the census.
One of the biggest assumptions that the Census Bureau has chosen to make is that the results of the survey, when compared to the results of the census, are always correct. The Bureau assumes that census is always wrong and is considered the source of all “erroneous” information even if the census taker “looked and sounded like” the person answering the door and was successful in securing cooperation from someone who might otherwise have refused to participate. In contrast, consider that the A.C.E. survey taker, probably from outside of the community, was unable to secure that same cooperation at a later date and gets a different set of answers. The Bureau’s adjustment methodology considers the census enumeration — even though it is, in reality, correct — as an “erroneous enumeration.”
We know that this assumption, in reality, is not always true. Consider that Census 2000 relied on recent immigrants and Alaska Natives during the enumeration to ensure an accurate count in their communities. These persons have greater affinity with their communities than A.C.E. survey takers, who were often recruited from outside of the community and it can reasonably be expected that they had greater success reaching out to members of their communities who fear or mistrust government. Yet the Bureau’s methodology assumes that the survey is always right. This assumption of the model is problematic and open to question.
4. Limitations of the Post-Strata
The methodology makes assumptions about the potential undercount of millions of people living in an increasingly diverse nation. The post-strata represent limited statistical assumptions about thousands of diverse racial and ethnic communities living in the United States.
For instance, the Bureau has to assume that a Mexican immigrant who has been in this country for less than eight months working in the strawberry fields of Washington and Oregon "statistically" has the same undercount rate as an American citizen of Mexican heritage whose family has lived in the United States for 150 years. (The Bureau assumes that they are statistically the same because they both are Males, Non-Owners, age 18-29, living in small rural towns.) Similar assumptions are repeated throughout the country and for all races.
5. MOST Data Used to Adjust is from ANOTHER State, City, Town
MOST of the data used to adjust in a particular state, city or town is from other states, other counties and other cities. In the A.C.E., undercount rates and adjustment factors were determined by COMBINING DATA from across the nation for some post-strata and from across a region for other post-strata
- Some people in Washington, DC would be adjusted with data from Baltimore, Atlanta, Charlotte, and Miami.
- Some people in Dallas and Houston, Texas would be adjusted with data from Chicago, Detroit, Los Angeles, Seattle and New York.
- Some people in Midland, Texas would be adjusted with data from not only Amarillo and Longview, but with data from Monroe, Louisiana; Bismarck, North Dakota; Cedar Rapids, Iowa; Boulder, Colorado; and Stillwater, Oklahoma.
6. Nullifying the Count of People who Chose to Participate in Census 2000
Statistical adjustment is not solely about adding people. It is also about subtracting people from the census count. Based on the adjustment factors assigned to the post-strata, millions of people are added or eliminated from the count by statistical adjustment. In 1990, the census would have subtracted about 1.4 million persons from the census — and not all of these people reflect the notion of the "undercounted."
Preliminary analysis in 2000 indicates that approximately one million people who filled out their census forms would have had their records nullified in order for the adjustment methodology to work properly. Not just Whites could be nullified under the 2000 adjustment. Asian children, for instance, were more likely to be subtracted than added under adjustment in 2000. Moreover, some Black, Hispanic and Asian homeowners were subtracted in 1990 and the preview of the 2000 data indicates it could happen again.
7. Correlation Bias
Why should anyone believe that an undocumented immigrant who refused to answer the questions of a census taker would have cooperated with another Census Bureau employee, the A.C.E. survey taker? If people are afraid of or hiding from the government, there is NO reason to believe that they are more likely to cooperate with the survey taker than a census taker.
This phenomenon is “correlation bias.” The statistical community, including the Census Bureau and the National Academy of Sciences, have ALWAYS acknowledged correlation bias as an impediment to the ability to improve the census through statistical adjustment.
8. Missing Data
Every survey, including the A.C.E. and the PES in 1990, contains a certain amount of missing data — data that is not secured during the survey process. The lack of such data means that data must be made-up, or “modeled,” and is not based on actual observation.
In other words, under adjustment the enumeration status of several million persons will be made-up based on modeling assumptions or otherwise unverifiable assumptions.
9. Statistical Adjustment is Not “Correction”
Statistical adjustment does not "correct" the census, not the way reasonable people expect. Most people believe that adjustment will correct the undercount — meaning if you have a neighborhood that was undercounted at a rate of 37 percent, the adjustment puts that 37 percent back in the same neighborhood. This is not the way adjustment works.
Statisticians and members of the National Academy of Sciences Panel to Review the Census, who were asked to review the A.C.E. methodology at a public meeting last October, took exception to the use of the term "statistical correction." It is because those scientists understand that there is no methodology available to the Census Bureau today that will "correct" the undercount and return the people who were missed in the census for Blacks, Latinos, American Indians, Asians and Pacific Islanders to the neighborhoods where they actually live.
What happens to the severely undercounted neighborhoods in minority communities as a result of adjustment?
In 1990, analysis of the methodology has proved, those neighborhoods having the most severely undercounted census blocks were only marginally improved. Even more disturbing was that some of the most overcounted census blocks actually received more people from adjustment — worsening the disparity between the census haves and have-nots. In fact, the higher the undercount, on average, the less improvement from adjustment. Former Chair of the National Academy of Sciences “Panel on Census 2000 Requirements in the Year 2000” and Beyond and a former adviser to President Jimmy Carter, Charles Schutlze told the US Commission on Civil Rights, “You shouldn`t use any of those numbers,” meaning adjusted numbers, “at the block level. There will be a lot of errors and probably some added errors at the block level because of the sampling problem.”
Further analysis, conducted by Charles D. Jones, the former Associate Director for the Decennial Census at the Census Bureau, on behalf of the Congressional Members of the US Census Monitoring Board, showed a similar pattern at the level of congressional districts. He has shown that congressional districts with high undercounts remain highly undercounted after adjustment. Undercount does not cancel out when the blocks are added together to form congressional districts because the undercount is not randomly distributed; rather it is concentrated in specific neighborhoods and communities. In other words, the disparity between census haves and have-nots does not go away.
Political representation and government funding are distributed according to geographic areas — communities and neighborhoods — not demographic groups.
However, many people assume that adjustment finds the persons who were missed and restores the missed people to the neighborhoods and communities where they were missed. The Associated Press reported yesterday of clinic workers in Brownsville, Texas who were concerned with the local census count and who believed that adjustment would account for all of the missed people. We know that adjustment does not put all of the missed persons back into the communities and neighborhoods where they were missed.
A Real Life Example
A real life example excerpted from the 1990 Post Enumeration Survey (the PES methodology is basically the same methodology as the 2000 A.C.E.) data for Houston, Texas illustrates the tale of two neighborhoods. The PES revealed a slight overcount in the area of Rice University, a predominantly white and affluent area of the city and it also indicated a severe, 45 percent, undercount in a Third Ward census block, a traditionally undercounted African American and low-income area.
What happened as a result of adjustment? Approximately the same numbers of people were added to each area — even to the overcounted area. The result was an even higher overcount of six percent in Southhampton and a reduction of the undercount in the Third Ward to 41 percent. This scenario would have been repeated throughout the country in 1990 — it happened in Chicago`s Bronzeville, near the public housing project Robert Taylor Homes, neighborhood. The heavily undercounted block received far less from adjustment than was needed, while adjustment added population to overcounted neighborhoods such as Lincolnwood, an area on Chicago`s north side.
Was it fair to over-adjust the Rice Village area merely because the methodology adjustment factors assigned to post-strata methodology demanded that it be done to achieve accuracy at the national, state or mega-city level? The adjustment methodology is not precise enough to put all of the people in the neighborhoods and census tracts were they are truly missing. You should ask the people living in Houston’s Third Ward if it is acceptable that they didn`t get the "correction" that they thought adjustment was going to give them.
However local governments do depend on accurate data and have an expectation that their data should be reliable. When we have asked local school board members, council members, or county commission members what is more important to them a) accurate neighborhood data for their constituents or b) a theoretically accurate number for the nation, state or group of 650,000, the answer has been overwhelmingly point “A.”
10. Side-Effects
Statistical adjustment should be understood as a therapy for the undercount, not a cure. Reasonable people will disagree about the right course of action, just as physicians have different ideas about the right course of medical treatments for arteriosclerosis or allergies. Like medical therapies, there can be more than one answer.
As with any therapy, statistical adjustment may have dangerous and unexpected side-effects. Before we embrace any treatment as a miracle cure, we ought to study, evaluate and assess the side-effects very carefully. This is exactly what the statisticians on the ESCAP committee have done and will continue to do. And if it had been an easy decision to make, the Bureau would have unanimously and strongly recommended the use of adjusted data.
The ESCAP’s recommendation not to use adjusted data at this time does not preclude the use of adjusted data for purposes other than distributing political representation. Whether such a policy decision would be wise is an issue to study for a later date. Adjusted data may be considered and may be appropriate if the adjusted data is shown as more reliable at appropriate levels — such as counties and states where many Federal funding decisions are based. Statistically adjusted data for funding purposes may make sense. The higher the level of geography — i.e., cities, counties, and states — statisticians believe that adjusted data becomes more reliable. On the other hand, at lower levels of geography — city blocks, census tracts, census blocks, and neighborhoods — statistically adjusted data is less reliable. We also note that while no western democracy uses adjusted census estimates for the purposes of distributing political representation, Australia and Canada — two countries with highly respected statistical agencies — do use adjusted data for other purposes, including the distribution of government funding and seem pleased with the results.
Conclusion: A Successful Census
Finally, it is important to realize that Census 2000 may be the most successful census in the history of this nation. We are closer to knowing how many people are living in America and where they are than anyone ever predicted or believed we would be. The Congressional Members of the Census Monitoring Board have always believed that the definition of success for the census in 2000 was a dramatic reduction in the differential undercount.
The Black undercount was reduced by 67 percent, from 4.5 in 1990 to 1.6 in 2000, and the Hispanic undercount was reduced by 60 percent, from 4.9 in 1990 to 2.2 in 2000. These reductions are unprecedented in census history. The differential undercount was substantially reduced and the Census Bureau deserves recognition and praise for this accomplishment. One of the reasons for this is the extra effort taken by the Regional Directors and their teams in hiring people who, as the Monitoring Board encouraged, looked and sounded like the people answering the doors in America’s neighborhoods.
During the coming months, the Bureau will continue to review Census 2000 and A.C.E. data and much more will be learned about this process. Both the Monitoring Board and the Bureau will examine these data very closely in order to form recommendations for 2010 so that the Bureau can finish the job they started in 2000 by reducing the differential undercount. The ultimate goal of everyone should be the elimination of the differential undercount through an accurate and fair census that counts every person at the correct address.
However, it is also important to understand the limitations of any adjustment methodology and to ensure that statistical “correction” at the block level is understood as a false promise. Statistical adjustment is a complex mathematical process that we know does not distribute undercount adjustment accurately at the block level. It may be appropriate to use these data for Federal funding purposes at the city, county or state level. Yet, the most important requirements of the census — to count every person accurately and at the correct address — are not met by the process. No methodology available to the Census Bureau today is capable of doing that.
###
|
|
