Content uploaded by James K Scott
Author content
All content in this area was uploaded by James K Scott on Dec 18, 2014
Content may be subject to copyright.
Cox and Scott
National Conference on Health Statistics
August 1999
1
MEASURING THE ECONOMIC IMPORTANCE OF HEALTH CARE IN
RURAL COMMUNITIES: the Role of Local Stakeholdersi
Anna M. Cox and James K. Scottii
Community Policy Analysis Center
University of Missouri
Involving stakeholders in community projects has captured increasing interest through out the 1990s
(Cf., Priscoli, 1995; Roberts, 1995; Roberts and Marshall, 1996; Sprecht et.al, n.d.; L.L. Decker and
Associates, Inc., n.d.). In the last ten years, many actors from private industry (Cf., Yosie and Herbst,
1998; International Finance Corporation, 1998) non-profit organizations (Aspen Institute, 1996) and
the scientific community (Stern and Fineberg, 1996) have strongly endorsed stakeholder involvement
for a wide range of applications. Several federal agencies such as the U.S. Department of Energy
(1994) the EPA (Fox, 1998) and others (Bierle, 1999) have also implemented comprehensive policies
and procedures for public involvement in community research projects, environmental impact
statements, regulatory review, and community planning. Clearly, whenever events or decisions
involve something new, something large and something very different in someone’s backyard,
stakeholders are now likely to be involved1.
However, despite the increased interest and commitment to stakeholder involvement, research on the
impact of this change is very limited. Evaluation of how stakeholder input affects problem definition,
data quality, research design and analysis, and community project outcomes is lacking. This paper
explores the role and effects of local stakeholders in community health research. It compares two
separate studies of the economic importance of health providers in a particular county in rural
Missouri. One study was conducted with no local involvement, using data from official government
sources. The second study was designed and completed by a team composed of social scientists and
local stakeholders using both primary and secondary data sources. The paper attempts to assess the
benefits and costs associated with stakeholder involvement. It begins with a review of recent research
and policy developments in stakeholder and public involvement. Next, it provides an overview of the
rural county selected for analysis, and compares the methods of both economic studies. Then, both
research findings and community reactions to each study are compared. It concludes with a list of
lessons learned, and a brief series of recommendations for future research.
1.0 Stakeholders and the Economic Importance of Health Care in Rural
Communities
At regional and state levels, agencies responsible for oversight and support of local health care
services have incorporated stakeholder involvement in community health planning for many years
(Cf., Missouri Department of Health, n.d.; Amundson, et.al, 1991; McGinnis, n.d.; Oklahoma Rural
Research and Demonstration Center, 1995). Because access to appropriate health care is fundamental
to the quality of life, it is clear that rural residents have much to gain from understanding and engaging
the issues that affect health care in their communities. However, recent changes in market conditions
and in federal and state health policy have contributed greatly to the complexity – and volatility – of
the health care system. Competition and cost pressures drive major changes in industry structure,
including organizational forms, insurance products, location and type of service providers, as well as
consumer choices and cost. Even though local health care is of critical importance, many consumers
find these issues very difficult to engage.
1 For a useful analysis of the social factors influencing the growth of public participation and stakeholder
involvement in community decisions, see Yosie and Herbst, 1998.
Cox and Scott
National Conference on Health Statistics
August 1999
2
These difficulties are often exacerbated in rural communities. In many rural areas, health providers
not only provide critical services to residents across large distances. They also provide some of the
most – and highest paying – jobs in the regional economy. Market and policy changes now threaten
the survival of many rural hospitals (The Lewin Group, 1999) and other providers (RUPRI, 1999).
This combination of circumstances underscores the need for rural residents to get more involved in
local health care decisions.
Operation Rural Health Works
In an effort to involve more rural residents in health care planning, and to demonstrate how health care
changes are affecting rural communities, the Federal Office of Rural Health Policy (US HHS) initiated
a pilot project called Operation Rural Health Works (ORHW). With support from the federal office,
the USDA, and the Rural Policy Research Institute (RUPRI), the project involves regional scientists
and Directors of the Office of Rural Health in five states2. The purpose of Operation Rural Health
Works is to expand public awareness of the importance of the economic impact of the health care
sector and to stress its critical role in rural development. Armed with this knowledge, it is anticipated
that local stakeholders will then become proactive and involved in planning and supporting their local
health system (RUPRI, 1999). The project was driven by three key research questions:
• How many people do health providers employ in rural communities?
• How are these jobs connected to other jobs in the region?
• What do health providers contribute to the overall regional economy?
To address these questions researchers are using secondary data and a standard method to assess the
economic importance of health care in every county of the five pilot states. In Missouri, this method
was applied to 115 counties, (93 of which are rural or non-metropolitan). When all the analysis of
county data is complete, project leaders will engage state and local stakeholders to review results and
assist in information dissemination and follow up at the local level. This has not occurred to date in
any of the states; so the impact of this information on community health planning is unknown.
However, community stakeholders were not involved in any of the research design or analysis of
results.
The Howell County Health Care Partnership
At about the same time that work began on Missouri’s Operation Rural Health Works, a citizens group
in Howell County, Missouri contacted the authors requesting a community-based assessment of the
same research questions. Howell County is in the middle of the Ozark Mountains on the Arkansas
state line. Once an agricultural region, its economy is now based on manufacturing, retail trade and
services. The County has attracted a number of early retirees in the last ten years, so its population
(35,000) is growing – and aging – quite rapidly (Cox, et.al, 1998). The County is also particularly
remote. In fact, the county seat is over 100 miles from the next larger place in any direction. With all
these characteristics, Howell County has become a medical center for residents and employers in a
thirteen county area across two states. When market and policy changes began to threaten major
health providers, community leaders wanted to know more about what was at stake for the regional
economy.
2 The five states involved in the pilot study include Kentucky, Missouri, Nevada, Oklahoma, and Pennsylvania.
Cox and Scott
National Conference on Health Statistics
August 1999
3
Because the issues were of significant concern to both residents and employers, we recommended
incorporating stakeholder involvement in this research3. Citizens were asked to convene a community
advisory panel (CAP) with representatives from all major institutions and interest groups. After some
deliberation, county residents chose sixteen members for the CAP, including chief executives from the
hospital, major employers, and a wide range of local organizations. The CAP took a very active role
in all phases of the research, and eventually decided to continue to meet regularly after the completion
of the project to discuss community health issues. The group adopted the name of the “Howell
County Health Partnership”.
In this instance, the goals of involving stakeholders were clear: 1) enhance the quality of information
available; 2) widen access of information to all interested parties; and 3) broaden participation in
public actions related to health care. CAP members agreed to play the following roles: 1) review
research objectives, methods and results to assure accuracy and relevance of findings; 2) foster
community interest and input into the project; and 3) assist in assessing and interpreting research
results. A process for involving stakeholders in the research was designed to address a series of
standard questions (Roberts and Marshall, 1996).
To what extent are local stakeholders (CAP members)?
• Experienced in public involvement processes?
• Informed on the issues pertinent to the project?
• Either hostile or apathetic to the aims of the project?
• United or divided in terms of interest and convictions?
• Representative of all key interests and actors in the community?
• Oriented toward local, regional and/or national perspective on project issues?
Interviews and focus groups were conducted with CAP members to complete this assessment.
Participation from all involved in the partnership remained high throughout the project.
2.0 Research Design
The coincidence of both studies affords an unusual opportunity to compare both the research findings
and the process involved. Both studies were completed by researchers at the Community Policy
Analysis Center (CPAC) – University of Missouri-Columbia. This section describes a detailed
methodology for both projects.
ORHW Data
The first study of employment and income in health care followed a method first described by
Doeksen, et.al. (1996). In this method, the definition of health care employment uses the Standard
Industrial Classification (SIC) Manual (U.S. Department of Commerce, 1987). The SIC codes have
four levels of disaggregation. The major industry groups are 1-digit codes. Within each major
industry group, the industry is subdivided into 2-, 3-, and 4- digit SIC levels. Each disaggregated level
3 Roberts (1995) recommends stakeholder involvement under the following conditions:
• When reaching a decision requires choosing between important social values.
• When results of a decision will significantly affect the interests of some people or groups.
• When the public perceives it has a lot to gain or lose by a decision.
• When the issue is already a source of some local controversy.
• When considerable environmental or social impact is expected.
• When public support is needed to implement a decision.
Cox and Scott
National Conference on Health Statistics
August 1999
4
adds up to the next-higher classification if all data are disclosed. That is, if all data are displayed, the
sum of the data in the 3-digit SIC codes will equal the 2-digit SIC group. For health services, the 2-
digit SIC code is 80. The sum of the 3-digit SIC codes within health services (i.e., 801, 802, 803, …,
809) will add up to the data for SIC code 80.
The health care sectors defined for this study are at the 3-digit SIC level, titled hospital (SIC 8060);
physician, dentist, and other professionals (SIC 8010-8040); nursing and protective care (SIC 8050);
other medical and health services (SIC 8070-8090); and pharmacies (SIC 5910, miscellaneous retail).
Although not classified as a health care service, pharmacies can be considered part of the health care
industry as they are made up of firms engaged in the retail sale of health care prescriptions and
products.
For the ORHW study, researchers in each state have the freedom to use the data and methods that are
the most current, reliable, and accurate for the counties in their state. Employment and income data
can be gathered from many different secondary data sources. An annual series was desired so year to
year changes could be calculated. Obtaining employment and income in the same data source was
also necessary in order to maintain consistency. County Business Patterns (CBP) is considered one of
the most reliable sources to obtain county-level employment and income data by disaggregated
industrial sectors. The data are compiled from several programs within the U.S. Census Bureau, as
well as from the administrative records of the Internal Revenue Service, the Social Security
Administration, and the Bureau of Labor Statistics (U.S. Bureau of the Census, 1999). The CBP data
are not subject to sampling errors because they are tabulated from universe files of the contributing
agencies. However, the data are subject to nonsampling errors. Nonsampling errors include such
things as conflicting definitions between contributing agencies, differences in interpreting questions,
or errors in coding, although due care is taken to minimize errors Statistics (U.S. Bureau of the
Census, 1999).
For employment and income figures, CBP includes full- and part-time employees who are on the
payroll in the pay period including March 12. Several categories of employed persons are not counted
in CBP, most significant is the exclusion of self-employed persons, a common occurrence in
secondary data sources. The most current data from County Business Patterns are for 1996.
The main limitation encountered when using CBP is the existence of nondisclosed data. No data are
published that would disclose the actual employment or payroll of an individual employer. When data
are not disclose, CBP replaces the employment data with a letter code (A-M) which indicates a range
of employment for that SIC code. Not only are the ranges for each code very large, but the
employment range is not consistent for each letter code. Instead, the size of the range increases for
each letter further in the alphabet. The ranges are defined as:
Cox and Scott
National Conference on Health Statistics
August 1999
5
Table 1. County Business Patterns Employment Ranges
Letter Code Employment Range Letter Code Employment Range
A1-19 H2,500-4,999
B20-99 I5,000-9,999
C100-249 J10,000-24,999
E250-499 K25,000-49,999
F500-999 L50,000-99,999
G1,000-2,499 M100,000 or more
Having nondisclosed data is a particular problem when working in less urbanized or populated
jurisdictions. In Missouri, 93 of the 115 counties are considered non-metropolitan counties. Most of
these counties, including Howell County, had nondisclosed data for the health care sectors.
Usually, there is very little to be done if data are not disclosed. Many researchers and analysts simply
take the midpoint of the employment range for each letter code as the estimated employment for that
sector. This is a valid, though simplistic, way of estimating employment for any particular sector.
The error in the employment estimate when using the midpoint can potentially be as large as 50
percent of the difference between the lower and upper limits of the range. This can be substantial
since the limits of each range are so wide. Data analysis also becomes problematic when the sum of
the parts does not add up to the whole. This lack of consistency can also confuse the clients for whom
the research is conducted.
However, CBP includes in their report establishment data, including the number of establishments in
different employment size classes. The employment size classes are:
Table 2. County Business Patterns Size Class Categories
Size Class Number of Employees Size Class Number of Employees
11-4 6100-249
25-9 7250-499
310-19 8500-999
420-49 91,000 or more
550-99
A more rigorous estimate of employment can be made, because of the structure of the Standard
Industrial Classification codes, by using both the given employment ranges and the employment size
class data. This procedure was developed by Kreahling, Smith, and Frumento (1996) and has been
used at the Pennsylvania State University for all the analysis that uses CBP data.
The resulting employment figure resulting from using the following procedure is still an estimate.
However, the size of the error will be less than when using the midpoint of the range given by CBP, as
the range is narrower and all available information is being used to determine a better estimate. Also,
following this procedure for estimating nondisclosed data ensures data consistency, where lower level
SIC codes sum to the next higher level SIC code.
Cox and Scott
National Conference on Health Statistics
August 1999
6
Estimating Nondisclosed Data in CBP
The first step is to sum the disclosed data at the 2-digit SIC level. Subtract the total of the disclosed
data from the total employment at the 1-digit SIC level. This is the total of the nondisclosed data.
The next step is to find the true employment range of each non-disclosed 2-digit category, using the
establishment size class data. The true minimum employment is calculated by multiplying the number
of establishments in each size class by the lower limit of each size class and summing the results for
the 2-digit SIC codes with nondisclosed data. The true maximum employment is calculated similarly,
where the number of establishments is multiplied by the upper limit of each size class and then
summed. The true minimum and true maximum are considered the lower and upper boundaries of the
true employment range.
For example, for the 1-digit SIC code for services (8) in Howell County there are two 2-digit SIC
codes with nondisclosed data in 1996--SICs 82 and 89. Total employment at the 1-digit SIC level for
services is 3,401. Total disclosed data sums to 3,369 leaving 32 employees to be accounted for by the
two nondisclosed SIC codes. The range for SIC 82 is given as range B, having 20 to 99 employees.
The true employment range for SIC 82 is calculated as follows (Tables 3 and 4):
Table 3. True Minimum of the Range for SIC 82
Size
Class
Number of
Establishments
Minimum Number
Employees in Size Class
11 * 1 = 1
32 * 10 = 20
Sum = 21
Table 4. True Maximum of the Range for SIC 82
Size
Class
Number of
Establishments
Minimum Number
Employees in Size
Class
11 * 4 = 4
32 * 19 = 38
Sum = 42
The true employment range for this nondisclosed 2-digit SIC is 21 to 42 employees. The size of the
range has been reduced by 75 percent.
The range for SIC 89 is given as range A, having 1 to 19 employees. The true employment range for
SIC 89 is calculated as follows:
Table 5. True Minimum of the Range for SIC 89
Size
Class
Number of
Establishments
Minimum Number
Employees in Size
Class
11 * 1 = 1
Sum = 1
Cox and Scott
National Conference on Health Statistics
August 1999
7
Table 6. True Maximum of the Range for SIC 89
Size
Class
Number of
Establishments
Minimum Number
Employees in Size
Class
11 * 4 = 4
Sum = 4
The true employment range for SIC 89 is 1 to 4 employees. The range size has been reduced by over
80 percent.
The true employment range usually gives results that narrow the range compared to that initially given
by CBP. There may be instances, though, where the true employment range falls above or below the
limits of the range indicated in CBP. When this occurs, the CBP range limits are to be used as
necessary. After completing this step, the true employment range is renamed the limited employment
range.
The next step it to calculate a new midpoint between the limited minimum and limited maximum for
each nondisclosed 2-digit SIC and sum them. To ensure that the sum of the estimated employment
equals the nondisclosed total, the midpoints must be adjusted further. This is done by calculating the
proportion of each midpoint with respect to the sum of the midpoints. This proportion is then
multiplied to the total of the nondisclosed data at the 1-digit SIC level. Using the Howell County
example:
Table 7. Final Employment Estimates for SIC 82 and SIC 89
SIC Midpoint of
Limited Employment Range
Proportion Adjusted
Employment Estimate
82 21 89.36% 29
89 2.5 10.64% 3
Sum of
Midpoints = 23.5 Total of
Nondisclosed data = 32
The final employment estimated should be checked for validity. In the Howell County example, the
resulting employment estimates fall with the ranges indicated by CBP. When the procedure is done
correctly, this should always be the case.
This procedure is used to estimate the employment levels for all nondisclosed data. For the 3-digit
SIC codes another iteration of this procedure is required, using the 2-digit SIC data obtained in the
first iteration as the total employment to which the 3-digit SIC sectors need to sum.
The procedure is detailed, but the final estimates for the lower level SIC categories are consistent with
the total employment at the next higher SIC level and provide less variability in the final employment
estimates. The use of this estimation procedure also reduces the absolute size of the potential error
caused by using the original midpoint. With smaller ranges, the maximum potential error in the
employment count is reduced.
When employment is not disclosed, neither is payroll. Unlike employment, though, no additional
information is disclosed for payroll data. Roughly 1/2 of the health care payroll at the 3-digit level is
non-disclosed for Missouri counties.
Cox and Scott
National Conference on Health Statistics
August 1999
8
Table 8. Disclosure of Payroll Data for Health Care Sectors
Health Sectors
SIC
Number of Counties
Without Payroll Data
Number of Counties
With Payroll Data
Average Payroll
per Employee
8000 3 112 $18,503
8010 42 58 $39,761
8020 52 59 $20,647
8030 57 21 $31,481
8040 46 50 $18,993
8050 33 74 $12,125
8060 64 10 $26,817
8070 25 15 $22,796
8080 34 18 $16,978
8090 43 16 $26,833
5910 54 54 $17,986
To obtain an estimate of health care sector payroll, the average pay per employee was calculated for
each 3-digit SIC code with disclosed payroll. The average annual pay is multiplied by the
employment for that SIC to obtain an initial estimate for total annual payroll. At this point, the sum of
the lower level SIC categories do not equal that of the next higher level. The difference between the
sum of the estimated payroll at the 2-digit SIC level and the total payroll at the 1-digit SIC level is
distributed according to each 2-digit SIC's proportion of the total estimated payroll. It should be noted
that the estimates of the nondisclosed payroll data are less reliable than are the employment estimates
because there is no counterpart to establishment data to furnish additional information.
Howell County Health Partnership Data
The health care sectors defined for this study closely follow the previously defined SIC categories.
That is, the main health sectors are hospital (SIC 8060); physician, dentist, and other professionals
(SIC 8010-8040); nursing and protective care (SIC 8050); other medical and health services (SIC
8070-8090); and pharmacies (SIC 5910, miscellaneous retail). In addition, the scope of what was
defined the health care sector was expanded to include those working in health care occupations,
regardless of industry. For example, school nurses are not counted as employed in the health care
industry, because education facilities are classified as government.
The differences between the two studies are the data sources used and the level of stakeholder
involvement in the process. In the 1997 Howell County study a survey instrument was prepared and
mailed to the entire population (vs. a sample) of firms providing local health care services to the
community. This survey elicited the number of full- and part-time jobs and total payroll from local
health care providers. In all, 79 different organizations provide health care to area patients. The
survey response rate of 97 percent was exceptionally high for a voluntary economic survey. This high
level of return indicates the level of community support for the project, and suggests that data
collected from the survey are highly reliable.
The survey response would not have been this high had the community and local stakeholders not
been involved in the process. Members of the health care advisory panel not only helped create the
survey instrument, but they also participated in the administration of the survey. Local residents
volunteered to make follow-up calls and provide technical assistance. The involvement of the
community is what made the survey return rate so high, which in turn led to highly reliable data.
In each study, once employment and income data were gathered for the five defined health care
sectors, input-output (I/O) modeling was used to estimate the total impacts of the health care sectors.
I/O provides a framework for measuring the linkages among sectors in a region's economy. The
Cox and Scott
National Conference on Health Statistics
August 1999
9
model is based on observed economic data for a specific geographical area (e.g. a county, state, or
nation). The transactions table, the basis of the input-output system, keeps track of the flow of goods
from each sector to other sectors and the final consumers (Miller and Blair, 1985). The flow of one
sector's output to other industries reflects the inter-sectoral linkages in an economy. The sum of these
linkages, added to the direct impact of a sector, is often called “the multiplier effect”.
I/O calculates several types of multipliers (e.g., income, employment, output, and value-added) for
each sector in a regional economy. Income and employment multipliers were used in both health care
impact studies. Multipliers measure linkages by including the indirect and induced employment and
income effects, for the respective type of multiplier. For the health care sectors, indirect effects refer
to the purchases of goods and services local health service firms make from other local businesses.
Induced effects occur when health service employees spend part of their earnings in local stores and
shops. In rural health care, the induced effects typically contribute the most to the overall economic
impacts. In Table 9, both employment and income multipliers are calculated for each of the relevant
sectors in Howell County. For example, an income multiplier of 1.45 for the hospital sector in Howell
County, Missouri means for every dollar earned by employees in the hospital sector, 45 cents of
income is earned by employees in other county businesses. Similarly, an employment multiplier of
1.31 for the hospital sector in Howell County means for every employee in the hospital sector, roughly
1/3 of a job in another county business is being supported.
Health Employment Income
Sector Multiplier Multiplier
Hospital 1.45 1.31
Physician, Dentist, & Other Professionals 1.61 1.36
Nursing and Protective Care 1.29 1.34
Other Medical and Health Services 1.41 1.38
Pharmacies 1.23 1.34
Weighted Average Multiplier 1.41 1.33
Table 9. Howell County Health Care Multipliers
The multiplier effect was measured for each health sector in every Missouri county using IMPLAN, a
pre-packaged I/O model (Minnesota IMPLAN Group, 1994). IMPLAN4 contains comprehensive
national data that is used to estimate regional data on a county-by-county basis. This model allows the
researcher to specify the geographic region of interest. In addition, the industries included in the
model are disaggregated and closely correspond to the SIC Manual structure.
Aside from the employment and income for the health care sectors, the data used in the analysis came
from the same sources. Personal income estimates are from the Bureau of Economic Analysis (1998).
The components of personal income also come from this source. All population estimates for both
studies are from the U.S. Census Bureau (1998). Total employment for each county is from County
Business Patterns, a program of the U.S. Census Bureau (1999). Since this source excludes the
government sector, the 1996 Missouri Employment and Wages, which follows the Bureau of Labor
Statistics guidelines, was used to obtain government employment and income (Missouri Department
of Labor and Industrial Relations, 1997).
4 IMPLAN is an acronym that stands for “IMpact analysis for PLANning”.
Cox and Scott
National Conference on Health Statistics
August 1999
10
3.0 Research Results
A comparison of research findings is provided in Table 10.
Health CBP Survey CBP Survey
Care Direct Direct Total Total
Sector Impact* Impact Impact* Impact
Hospital 1,000 1,231 1,445 1,779
Physician, Dentist & Other Professionals 233 246 374 395
Nursing and Protective Care 621 747 801 964
Other Medical and Health Services 197 212 278 299
Pharmacies 61 75 75 92
Total 2,112 2,511 2,973 3,528
Hospital $20,803 $32,196 $27,188 $42,077
Physician, Dentist & Other Professionals $5,040 $13,513 $6,867 $18,414
Nursing and Protective Care $8,399 $10,791 $11,244 $14,447
Other Medical and Health Services $2,924 $4,260 $4,031 $5,873
Pharmacies $1,674 $2,035 $2,246 $2,731
Total $38,840 $62,796 $51,576 $83,541
CBP Survey
Average Pay Average Pay
Hospital $20,803 $26,154
Physician, Dentist & Other Professionals $21,629 $54,931
Nursing and Protective Care $13,525 $14,446
Other Medical and Health Services $14,843 $20,095
Pharmacies $27,441 $27,139
Average of All Health Care Sectors $18,390 $25,008
* Includes estimates when data were not disclosed
ANNUAL PAY PER EMPLOYEE
INCOME (000)
EMPLOYMENT
Table 10. Howell County Health Sector Impact on Employment and Income
A review of Table 10 shows that the primary data study found substantially greater economic impacts
than the study using secondary data. These discrepancies are due to at least three important
differences in data quality. First, secondary data disclosure issues explain most of the differences
between the two studies, and the estimation procedures described above. All but one of the key health
sectors for Howell County contained non-disclosed data. The primary of Howell County health
providers suggests that in each sector the estimation technique under-estimates both employment and
income. Second, the definition of health providers adopted by stakeholders included more jobs and
income than the standard method, including school nurses and public health workers that are typically
classified in the SIC manual as part of the government sector. Third, the local stakeholders adopted a
more comprehensive definition of income from health providers than that used in the CBP study. The
latter includes only labor or payroll income, whereas the County survey requested data on all earned
income – including proprietors’ income - from local health providers. This is most clearly seen in the
Physicians, Dentists and Other Professionals sector. Data from the County survey includes proprietors
Cox and Scott
National Conference on Health Statistics
August 1999
11
income for medical and dental practices, so the average income per job is over $30,000 more than
indicated in the secondary data study. Since most of the multiplier effects are due to the local
purchases of health care workers, this difference produces much greater total impacts – both in terms
of employment and local income.
The divergence between the two data sources of the employment and income measures are notable
(Table 11). Using the survey results as a base, CBP consistently underestimated the direct impact of
the health care sectors. Reductions in employment estimates reach 18.9 percent for the hospital
sector. The income measures showed even larger differences in the CBP study, underestimating the
impact of health care by 38 percent. The largest difference - 62.7 percent - occurred in the Physician,
Dentist, and Other Professionals sector. A portion of this difference is likely attributed to exclusion of
proprietor's income in CBP.
Health Care
Sector
Employment Income
Hospital -18.8% -35.4%
Physician, Dentist, & Other Professionals -5.3% -62.7%
Nursing and Protective Care -16.9% -22.2%
Other Medical and Health Services -7.1% -31.4%
Pharmacies -18.7% -17.8%
All Health Care Sectors -15.9% -38.1%
Percent Difference
Using CBP
Table 11. Reduced Estimates Using CBP
The results presented in Table 11 are of particular concern with respect to achieving and maintaining
stakeholder involvement in community health planning. The survey results, which showed that health
providers directly contributed almost 25 percent of total employment and twelve percent of total
personal income (TPI)5 motivated the stakeholders to continue their efforts in supporting the local
health care sector. Had the CBP results been presented instead, the level of subsequent public
involvement in community health planning may well have been less.
4.0 Lessons Learned
Comparing both the process and the product of the two Howell County studies - along with our
growing experience with stakeholder and public involvement in community projects – suggests a
number of key lessons learned.
• Stakeholder involvement enhanced the quality of data collection and analysis.
The Howell County survey generated an exceptional response rate (97%). With few exceptions,
the returned surveys were fully and accurately completed. We attribute the quality of survey
response to the direct involvement of the CAP in data collection. They worked to identify a
population list of organizations that provided a wide range of local health services. They also
identified individual contacts at each of these organizations, reviewed the survey instrument for
any confusing or misleading questions, contacted each potential respondent to request their
participation and offer assistance in completion of the instrument, and guarded the anonymity and
confidentiality of all respondents through out the process.
5 Total Personal Income includes earnings, dividends, interest, rent, and transfer payments.
Cox and Scott
National Conference on Health Statistics
August 1999
12
The benefits of stakeholder involvement extend beyond the quality of data for a study.
Convening a group of diverse stakeholders to discuss a critical community issue, in itself, fosters
public dialog, generates new ideas and community learning. It also helps to encourage broader
ownership and involvement in community issues. Obviously, none of this assistance was
available to researchers applying secondary data.
• Facilitating stakeholder involvement required significant resources.
It is also clear from this and other studies that stakeholder and public involvement in community
research is expensive. Effective stakeholder involvement requires careful assessment – both of
the issues, and of potential participants. It also requires significant planning, and background
research into the issues, the community and its surrounding region, and the individuals,
organizations and relationships represented in the project. The facilitator must be trained in group
dynamics, negotiations and dispute resolution. Once all the preparatory work is complete, the
project team has to manage communications with members and with media, arrange for meeting
facilities and equipment, prepare and distribute meeting packets, record, display and edit meeting
process notes, and travel to the site for multiple project meetings. When community work covers
a broad distance, travel expenses can quickly add up.
In our research center, we conduct 8-10 community-based research projects each year involving
stakeholders and focus groups through out our State. Although these projects vary in scope and
complexity, the non-fixed costs average about $15,000 each. We estimate that 50-60% of these
costs is incurred in stakeholder and public involvement.
Clearly, there are tradeoffs between the benefits and costs associated with the more participatory
approach. On the one hand, the quality of research conducted with community involvement is
improved. However, the human and other resource requirements limit the number of projects
completed, at a time when demand for community research is growing rapidly6.
• Conduct a comprehensive evaluation of stakeholder involvement for the project.
Until now, assessment of the outcomes and the process of stakeholder involvement in community
projects has been quite limited (Yosie and Herbst, 1998). Indeed, evaluation of this component of
our own center has been sketchy and informal, at best. However, this year we are implementing a
formal process for evaluating our work with community advisory panels. We believe that our
work with stakeholders is perhaps the most effective part of our community research program.
However, without comprehensive evaluation, we probably miss opportunities to learn and
improve, and we definitely miss opportunities to demonstrate impacts and secure more funding to
pursue our mission. The evaluation scheme we are developing will include a standard set of
procedures, as well as address the particular goals of each community project we conduct.
• Consider what happens to the stakeholder group after the completion of the project.
In most of our community projects, stakeholders, in the form of community advisory panels,
engage a specific issue for a designated length of time, and – when the project is complete, end
their association to move on to other interests. In the Howell County health care study, panelists
chose to continue to work together to apply what they had learned and stimulate further public
debate about the future of local health providers. Although we planned this project carefully, we
did not anticipate their collective decision to continue, and we did not budget adequate time or
resources to provide the continuing support they requested. Two years after completion of this
6 There are several sources of possible funds to support public participation. For example, see National
Partnership for Reinventing Government (1998).
Cox and Scott
National Conference on Health Statistics
August 1999
13
research, members of the Howell County Health Partnership continue to meet, and we remain in
touch, though resource constraints make it difficult.
5.0 Issues for the Future
Consultation with stakeholders is now, and will remain, a very important component of public
decision making and community research in all policy sectors at all levels of government. However,
scholars and practitioners of public involvement face a number of key issues as they work to improve
outcomes for the future. Several are listed below for consideration.
• Realistic expectations about the role stakeholder and public involvement can play in public
decision making (Yosie and Herbst, 1998)
Clearly, interest and commitment to enhanced stakeholder involvement is growing. This growth
is a reaction to increased public demand for involvement in public decision making, and increased
expectations about the role that the public can play in making difficult choices. In perhaps the
most comprehensive review of case studies of stakeholder involvement, Yosie and Herbst (1998)
conclude that the greatest challenge for all interested in enhanced public participation is to
anticipate heightened expectations, and to address them proactively.
• Better measurement of the impact of public involvement is needed.
As suggested above, one of the difficulties of responding to public expectations for increased
participation is in managing the costs associated with effective stakeholder involvement.
Community researchers and public officials must look for ways to improve both the process and
the outcomes of public participation. We must also find ways to justify using the resources
required to meet current and future demands. Improved impact assessment is critical for this
justification.
• Linking scientific research with public involvement.
There is growing evidence that citizens are increasingly skeptical of scientists and scientific
findings – particularly when research findings threaten their interests. Public consultation
combined with scientific knowledge can be extremely powerful in local problem solving.
However, more work is required to enhance communication between researchers and citizens. In
particular, scientists need to be convinced of the value of collaborating with community residents.
• Linking stakeholder and public deliberation with policy decision making.
It is counterproductive to engage stakeholders in specific issues if their input will not inform or
affect the decision or action they care about. As changes in governance and policy decision
making continue to evolve, it is increasingly important that mechanisms be developed to engage
citizens in authentic participation in policy decisions – including decisions about the maintenance
and use of public information systems.
• Better facilitation of stakeholder processes.
Despite the increase in interest, training materials and public support, managing effective
stakeholder involvement generally still needs major improvements.
Cox and Scott
National Conference on Health Statistics
August 1999
14
6.0 References
Amundson, Bruce, et.al
1991 Implementing a Community-Based Approach to Strengthening Rural Health
Services: the Community Health Services Development Model. WAMI Rural Health
Research Center. Dept. of Family Medicine, Research Section, University of
Washington – Seattle: Rural Health Working Paper Services, Working Paper #1.
February 1991.
Aspen Institute
1996 The Alternative Path: A Cleaner, Cheaper Way to Protect and Enhance the
Environment.
Beierle, Thomas C.
1998 “The Federal Advisory Committee Act and the Public Participation Paradox.”
Center for Risk Management Newsletter (14):3-4
Bureau of Economic Analysis
1998 1996 Regional Economic Information System. [Online]
http://www.fisher.lib.virginia.edu/reis/
Community Policy Analysis Center
1999 Operation Rural Health Works: Howell County Executive Summary.
Cox, Anna, James K. Scott, Anna Kovalyova, David Peters, Benjamin Winchester,
Thomas G. Johnson
1998 Howell County Baseline: 1998-2007. Community Policy Analysis Center Research
Report. May, 1998.
Doeksen, Gerald, Thomas R. Harris and Cheryl St. Clair
1999 Relationship of Health Care to Economic Development. Rural Action Series Vol. 1.
No. 1. Washington, D.C.: National Association of Counties. Rural Action Caucus.
July 1999.
Doeksen, Gerald, Thomas G. Johnson and Chuck Willoughby
1996 Measuring the Economic Importance of the Health Sector on a Local Economy: A
Brief Review of Procedures to Measure Local Impacts. Starkville, MS: Southern
Rural Development Center
Fox, J. Charles
1998 A Real Public Role. The Environmental Forum November, 1998. [Online]
http://www.epa.gov/reinvent/notebook/eli1198.htm.
International Finance Corporation
1998 Doing Better Business through Effective Public Consultation and Disclosure: A
Good Practice Manual. Washington DC: the World Bank.
Johnson, Thomas G. and James K. Scott
1998 The Changing Nature of Rural Communities. Increasing Understanding of Public
Problems and Policies 1997. D. Ernstes and D. Hicks, eds. Oak Brook, IL: The
Farm Foundation. Pp. 177-188.
Kleiner, Anna, James K. Scott, Thomas Johnson and Anna Kovalyova
1997 The Economic Importance of Health Care in Howell County, Missouri. Community
Policy Analysis Center Research Report. November, 1997.
Kreahling, Kathleen S., Stephen M. Smith, and Claudio Frumento.
1996 "A Procedure for Estimating Nondisclosed Employment in the County Business
Patterns Data Set." Department of Agricultural Economics and Rural Sociology,
Staff Paper 229. University Park, PA: The Pennsylvania State University.
L.L. Decker & Associates, Inc.
N.D. Community-Based Planning, Intervention and Involvement [Online]
http://www.lldecker.com/publicn.htm.
Cox and Scott
National Conference on Health Statistics
August 1999
15
McGinnis, Paul
N.D. Healthy Futures – A Development Kit for Rural Hospitals. Mountain States Health
Corp.
Miller, Ronald E., and Peter D. Blair.
1985 Input-Output Analysis: Foundations and Extensions. Englewood Cliffs, New
Jersey: Prentice-Hall, Inc.
Minnesota IMPLAN Group.
1996 IMPLAN Ver. 94-F: Minneapolis, MN.
Missouri Department of Health
N.D. The Community Health Assistance Resource Team. [Online]
http://www.health.state.mo.us/CHART/CHARToverview.html
Missouri Department of Labor and Industrial Relations
1997 1996 Missouri Employment and Wages. Research and Analysis Section, ES-202
Program. Jefferson City, MO: State of Missouri.
National Partnership for Reinventing Government
1998 Federal Support for Communities Initiative – A Multi-Agency Reinvention
Laboratory. [Online] http://www.npr.gov/initati/fedsuppt.html
O’Dubhchair, Kate, James K. Scott, and Thomas G. Johnson
Forthcoming (A) “Building a Knowledge Infrastructure for Learning Communities” in Michael
Gurstein, Editor. Community Informatics: Enabling the Community Use of
Information and Communications Technologies
O’Dubhchair, Kate, James K. Scott, and Thomas G. Johnson
Forthcoming (B) “Community Decision Support Systems: Managing Knowledge for Sustainable
Community Development” in G. David Garson, ed. Handbook of Public Information
Systems. New York: marcel dekker, inc.
Oklahoma Rural Research and Demonstration Center
1995 Guidebook on Strategic Health Planning for Oklahoma Communities. Oklahoma
Dept. of Health, University of Oklahoma Health Sciences Center and Oklahoma
State University Cooperative Extension Service.
Priscoli, Jerome Deli
1995 Twelve Challenges for Public Participation Practice. Interact (Fall) 1995: 78ff.
Roberts, Richard
1995 Public Involvement: from Consultation to Participation. In Environmental and Social
Impact Assessment F. Vanclay and D. Bronstein, Eds. John Wiley and Sons, Ltd.
Roberts, Richard and Nancy Marshall
1996 Stakeholder Consultation: Only One Component of Public Involvement! Praxis
Papers. Calgary, Alberta: Praxis, Inc.
Rural Policy Research Institute
1999 Operation Rural Health Works. RUPRI Policy Brief. [Online]
http://rupri.org/programs/orhw/index.html
Rural Policy Research Institute Health Panel
1999 Taking Medicare into the 21st
Century: Realities of a Post BBA World and
Implications for Rural Health Care. RUPRI P99-2. February 1999.
Scott, James K and Steven Henness
1999 "From Devolution to New Governance: A Review of Emerging International
Literature". Presented at the Annual Meetings of the Rural Sociological Society.
Chicago: August 1999.
Specht, Elaine, Andrew Walker, Gene Gardner and Diana Krop
N.D. Improving Communication to Improve Involvement: Research in Communication
and Stakeholder Involvement. [Online]
http://www.wpi.org/rcpi/articles/impcom.htm.
Cox and Scott
National Conference on Health Statistics
August 1999
16
Stern, Paul and Harvey Fineberg, eds.
1996 Understanding Risk: Informing Decisions in a Democratic Society. Washington,
DC: National Academy Press.
The Lewin Group
1999 The Balanced Budget Act and Hospitals: The Dollars and Cents of Medicare
Payment Cuts. [Online] http://www.aha.org/LweinTOC.html.
The Show Me Project
1999 The Economic Importance of Health and Social Services in Fermanagh. Derry,
Northern Ireland. The University of Ulster
U.S. Bureau of the Census
1998 Population Estimates Program. Population Division. [Online]
http://www.census.gov/population/www/estimates/popest.html
U.S. Bureau of the Census
1999 1996 County Business Patterns. Data in CD-ROM. Washington D.C.: US
Government Printing Office.
U.S. Department of Commerce
1987 Standard Industrial Classification Manual. National Technical Information Service.
Washington D.C.: US Government Printing Office. Also [Online]
http://www.osha.gov/oshstats/sicser.html
United States Department of Commerce.
1999 Falling Through the Net: Defining the Digital Divide. [Online]
http://www.ntia.doc.gov/ntiahome/fttn99/contents.html
United States Department of Energy
1994 Public Participation. Washington D.C.: DOE Policy 1210.1. July 29, 1994.
Westen, Tracy.
1998 “Can Technology Save Democracy?” [Online]
http://www.cgs.org/papers/techdem.htm.
Yosie, Terry and Timothy Herbst
1998 Using Stakeholder Processes in Environmental Decisionmaking: An Evaluation of
Lessons Learned, Key Issues and Future Challenges. September, 1998.
i This paper was presented at the National Conference on Health Statistics. Washington, DC. August 1999.
ii Anna M. Cox is Research Coordinator and James K. Scott is Program Director at the Community Policy
Analysis Center – University of Missouri-Columbia. Direct correspondence to:
James K. Scott, Ph.D.
Community Policy Analysis Center
143B Mumford Hall
University of Missouri
Columbia, MO 65211
Scottj@missouri.edu
http://www.cpac.missouri.edu
