|
Performance Measurement in Human Services Contracts The use of performance measures in public contracts with nonprofit service delivery organizations is a growing practice. This is particularly true in New York City where government has made a strong commitment to performance contracting. Performance measurement can demonstrate the quality of service delivered and ensure accountability by organizations that contract with government. It is important to analyze the contribution of this practice to both public understanding of the impact of services and to the effective functioning of the government-nonprofit system that typifies delivery of human services. Most City departments that contract with nonprofit service providers include performance indicators that must be reported on a regular basis. In some cases, payments to the nonprofit contractors are based on performance, while in others, specified indicators must be reported and do not affect payment on current contracts, although they may be factors in decisions about renewal and extensions of contracts. This basic distinction between performance-based contracts, in which payments to contractors are determined by performance milestones, and performance standards, in which performance measures must be reported but do not determine payments, is an important one. The development and implementation of performance contracts is done individually by each City department. It does not appear that there are any overall policies or guidelines. Neither contract rules issued by the Procurement Policy Board, the city agency that establishes contracting regulations under the City Charter, nor the requirements of the Mayor’s Office of Contract Services (MOCS), which manages the contracting process, create procedures for how performance measures are developed or implemented. Nor is there any central source for information about performance contracting. Neither MOCS, the Office of the Comptroller nor the Mayor’s Office of Operations track or report on the use of performance contracts and/or the measures used by departments in their contracts with nonprofit and for-profit providers of services. To get the information for our inquiry, we needed to go department-by-department and even program-by-program to determine the measures used, whether they are the basis for payment, how performance is documented, and how satisfied the City departments are with both performance and process. Performance-Based Contracts As a result, the nature of performance indicators, which are often developed at the program level within departments, vary widely, as do the milestones to be achieved and the means used to document performance. Generally, measures establish milestones that determine all or a portion of the payment that agencies receive under their contracts with City departments. Some contracts are “hybrid” or partly performance-based—meaning that a percentage of the payments to the contractor are based on meeting performance milestones, and a portion of the payments are paid according to the line-item budget for the contract. The Human Resources Administration (HRA), which administers the City’s welfare program, has the most extensive history with performance-based contracts. (See “HRA: Leading the Way” below.) The Department of Homeless Services, which was one of the first City agencies to develop performance incentives for paying its contractors, actively utilizes performance-based contracting to place people from homeless shelters into permanent housing. Current performance measures emphasize housing placement out of shelters for homeless individuals and families. Bonuses or penalties of 5% are awarded in the amount of the contract for the subsequent year following the documented performance. The Department of Homeless Services has also recently restructured and renegotiated its street outreach contracts to provide one contract for each of the five boroughs of New York City, including performance-based measures of reduced street homelessness and placement into permanent housing. And the department contracts through its HomeBase program with community-based organizations in areas of the City that have generated high numbers of homeless families to prevent such placements into City-funded shelters, basing payments partly on diversion of families from the City’s shelter entry center and partly on line-item budgets. Some City contracts for children’s services contain provisions for performance-based payments. For example, the Department of Youth and Community Development’s Out-of-School Time (after school) program, which began two years ago, requires documented enrollment and attendance, for which its contractors are responsible or subject to a penalty of 10%. The Administration for Children’s Services (ACS), which is the City’s public child welfare and child care department, has an elaborate evaluation system for rating foster care agencies. And its child care providers are becoming subject to full-enrollment/attendance requirements, for which they are at risk of payment reductions based on shortfalls. It is clear from this preliminary investigation that City departments are increasing the applications of performance milestones to the process for paying its human services contractors. Performance Standards Much more common than performance-based measures are performance standards that contracting agencies must report to City departments, but which do not constitute a basis for payment on their contracts. Performance according to such standards can affect renewal and continuation of contracts, but payments are based on the line-item budgets that have been negotiated. All City human services agencies require their contractors to report indicators of performance. These range from senior citizen centers under the auspices of the Department for the Aging, which report numbers of meals served, to various programs under the Human Resources Administration—including domestic violence, home care, HIV/AIDS, and adult protective services. What do Nonprofit Providers Think? For the most part, performance-based contract milestones and performance standards are prescribed by City agencies without extensive consultation with service providers. In some cases, City departments have been willing to modify provisions in response to concerns expressed by the contracting agencies, but there has not generally been a process to develop a consensus as to what milestones and indicators are appropriate. Nonprofit human services agencies generally support performance contracting, but have substantial reservations and concerns about the way it is being applied and implemented. It is clear from a Baruch College survey and from discussions with service providers that the reporting requirements are considered burdensome and agencies do not feel that they are compensated for the costs involved. Reporting and documentation requirements, even for similar programs, are not consistent among City departments, and where automated, generally use information data systems that are unique to each program. Thus, contracting agencies must develop separate response capabilities for each program area for which they contract. There is clearly greater risk for contractors under performance-based contracts, but the combination of the two kinds of requirements increases the administrative burden on agencies, especially those with multiple City contracts. Regarding the financial risks involved in performance-based contracts, agencies feel that there is always a downside for not meeting milestones, but there is little or no upside for performance that exceeds targets. Either there is no reward, or the reward is deferred into the subsequent contract year. Agencies want a system that rewards superior performance with flexibility that would allow funds to be transferred among programs that the agency administers, which is a practice that does not seem possible under current City contracting provisions. There are substantial numbers of nonprofit organizations that contract with several or many City departments and programs and would welcome a performance-based contract process that allowed funds to be shifted among programs. How Does Performance While the use of performance-based contracts and performance standards is expanding, there is not good documented evidence of which approach produces better results and under what conditions. The current pattern of separate department-by-department performance reporting does not generally enable City government to provide the public with broad assessments of progress toward major social outcome objectives regarding children, family and individual well-being. A reasonable question to ask is whether the individual indicators being measured can be accumulated to provide evidence of trends and status of poverty, family conditions, children’s security and development, employability of the workforce, socioeconomic mobility, and other over-arching social welfare goals. To do so would require agreement across City departmental and contracting agency spheres about appropriate social goals, what indicators should be utilized, and how to adjust current performance measures to yield specific data that could be accumulated to inform the City, service providers, and the public about progress being made. The following recommendations are intended to improve and support the practice of performance measurement in New York City human services contracting. Any changes should be incremental and should not undermine the system of accountability that has been established through the work done to date by City government. Recommendations: The City should consult widely with nonprofit human services provider agencies and the organizations that represent them to develop appropriate indicators of program-specific performance and, wherever possible, to develop common and consistent means to report and document them. Special attention should be paid to reporting requirements for agencies with multiple City contracts. The City and provider agencies should develop over-arching outcome measures that are supported by program-specific indicators and that can be used to assess progress toward achieving broad social goals. The administrative costs of these measurement and reporting systems should be identified and shared in an equitable way by the City and the contracting agencies. As New York State develops increased performance contracting requirements, City and State officials should coordinate their efforts with regard to performance indicators, means of reporting and documenting performance, and overall measures of social outcome goals. Finally, research should continue to complete the reconnaissance of City department performance contracting measures and practices. Discussions with major service providers and organizations that represent them should also continue, leading to case studies of the impact of performance contracting on service delivery and to further recommendations for improved practices in the joint City-nonprofit system that typifies New York City human services. In addition, work should be done to determine the direction of New York State in establishing performance measures in its contracts and to compare these requirements with those of the City. Foundations and corporate funders are also establishing performance measures that should be included in this analysis. Jack Krauskopf is Distinguished Lecturer and Director of the Center for Nonprofit Strategy and Management at Baruch College (City University of New York), School of Public Affairs. The use of performance measures in public contracts with nonprofit service delivery organizations is a growing practice. This is particularly true in New York City where government has made a strong commitment to performance contracting. Performance measurement can demonstrate the quality of service delivered and ensure accountability by organizations that contract with government. It is important to analyze the contribution of this practice to both public understanding of the impact of services and to the effective functioning of the government-nonprofit system that typifies delivery of human services. Most City departments that contract with nonprofit service providers include performance indicators that must be reported on a regular basis. In some cases, payments to the nonprofit contractors are based on performance, while in others, specified indicators must be reported and do not affect payment on current contracts, although they may be factors in decisions about renewal and extensions of contracts. This basic distinction between performance-based contracts, in which payments to contractors are determined by performance milestones, and performance standards, in which performance measures must be reported but do not determine payments, is an important one. The development and implementation of performance contracts is done individually by each City department. It does not appear that there are any overall policies or guidelines. Neither contract rules issued by the Procurement Policy Board, the city agency that establishes contracting regulations under the City Charter, nor the requirements of the Mayor’s Office of Contract Services (MOCS), which manages the contracting process, create procedures for how performance measures are developed or implemented. Nor is there any central source for information about performance contracting. Neither MOCS, the Office of the Comptroller nor the Mayor’s Office of Operations track or report on the use of performance contracts and/or the measures used by departments in their contracts with nonprofit and for-profit providers of services. To get the information for our inquiry, we needed to go department-by-department and even program-by-program to determine the measures used, whether they are the basis for payment, how performance is documented, and how satisfied the City departments are with both performance and process. Performance-Based Contracts As a result, the nature of performance indicators, which are often developed at the program level within departments, vary widely, as do the milestones to be achieved and the means used to document performance. Generally, measures establish milestones that determine all or a portion of the payment that agencies receive under their contracts with City departments. Some contracts are “hybrid” or partly performance-based—meaning that a percentage of the payments to the contractor are based on meeting performance milestones, and a portion of the payments are paid according to the line-item budget for the contract. The Human Resources Administration (HRA), which administers the City’s welfare program, has the most extensive history with performance-based contracts. (See “HRA: Leading the Way” below.) The Department of Homeless Services, which was one of the first City agencies to develop performance incentives for paying its contractors, actively utilizes performance-based contracting to place people from homeless shelters into permanent housing. Current performance measures emphasize housing placement out of shelters for homeless individuals and families. Bonuses or penalties of 5% are awarded in the amount of the contract for the subsequent year following the documented performance. The Department of Homeless Services has also recently restructured and renegotiated its street outreach contracts to provide one contract for each of the five boroughs of New York City, including performance-based measures of reduced street homelessness and placement into permanent housing. And the department contracts through its HomeBase program with community-based organizations in areas of the City that have generated high numbers of homeless families to prevent such placements into City-funded shelters, basing payments partly on diversion of families from the City’s shelter entry center and partly on line-item budgets. Some City contracts for children’s services contain provisions for performance-based payments. For example, the Department of Youth and Community Development’s Out-of-School Time (after school) program, which began two years ago, requires documented enrollment and attendance, for which its contractors are responsible or subject to a penalty of 10%. The Administration for Children’s Services (ACS), which is the City’s public child welfare and child care department, has an elaborate evaluation system for rating foster care agencies. And its child care providers are becoming subject to full-enrollment/attendance requirements, for which they are at risk of payment reductions based on shortfalls. It is clear from this preliminary investigation that City departments are increasing the applications of performance milestones to the process for paying its human services contractors. Performance Standards Much more common than performance-based measures are performance standards that contracting agencies must report to City departments, but which do not constitute a basis for payment on their contracts. Performance according to such standards can affect renewal and continuation of contracts, but payments are based on the line-item budgets that have been negotiated. All City human services agencies require their contractors to report indicators of performance. These range from senior citizen centers under the auspices of the Department for the Aging, which report numbers of meals served, to various programs under the Human Resources Administration—including domestic violence, home care, HIV/AIDS, and adult protective services. What do Nonprofit Providers Think? For the most part, performance-based contract milestones and performance standards are prescribed by City agencies without extensive consultation with service providers. In some cases, City departments have been willing to modify provisions in response to concerns expressed by the contracting agencies, but there has not generally been a process to develop a consensus as to what milestones and indicators are appropriate. Nonprofit human services agencies generally support performance contracting, but have substantial reservations and concerns about the way it is being applied and implemented. It is clear from a Baruch College survey and from discussions with service providers that the reporting requirements are considered burdensome and agencies do not feel that they are compensated for the costs involved. Reporting and documentation requirements, even for similar programs, are not consistent among City departments, and where automated, generally use information data systems that are unique to each program. Thus, contracting agencies must develop separate response capabilities for each program area for which they contract. There is clearly greater risk for contractors under performance-based contracts, but the combination of the two kinds of requirements increases the administrative burden on agencies, especially those with multiple City contracts. Regarding the financial risks involved in performance-based contracts, agencies feel that there is always a downside for not meeting milestones, but there is little or no upside for performance that exceeds targets. Either there is no reward, or the reward is deferred into the subsequent contract year. Agencies want a system that rewards superior performance with flexibility that would allow funds to be transferred among programs that the agency administers, which is a practice that does not seem possible under current City contracting provisions. There are substantial numbers of nonprofit organizations that contract with several or many City departments and programs and would welcome a performance-based contract process that allowed funds to be shifted among programs. How Does Performance While the use of performance-based contracts and performance standards is expanding, there is not good documented evidence of which approach produces better results and under what conditions. The current pattern of separate department-by-department performance reporting does not generally enable City government to provide the public with broad assessments of progress toward major social outcome objectives regarding children, family and individual well-being. A reasonable question to ask is whether the individual indicators being measured can be accumulated to provide evidence of trends and status of poverty, family conditions, children’s security and development, employability of the workforce, socioeconomic mobility, and other over-arching social welfare goals. To do so would require agreement across City departmental and contracting agency spheres about appropriate social goals, what indicators should be utilized, and how to adjust current performance measures to yield specific data that could be accumulated to inform the City, service providers, and the public about progress being made. The following recommendations are intended to improve and support the practice of performance measurement in New York City human services contracting. Any changes should be incremental and should not undermine the system of accountability that has been established through the work done to date by City government. Recommendations: The City should consult widely with nonprofit human services provider agencies and the organizations that represent them to develop appropriate indicators of program-specific performance and, wherever possible, to develop common and consistent means to report and document them. Special attention should be paid to reporting requirements for agencies with multiple City contracts. The City and provider agencies should develop over-arching outcome measures that are supported by program-specific indicators and that can be used to assess progress toward achieving broad social goals. The administrative costs of these measurement and reporting systems should be identified and shared in an equitable way by the City and the contracting agencies. As New York State develops increased performance contracting requirements, City and State officials should coordinate their efforts with regard to performance indicators, means of reporting and documenting performance, and overall measures of social outcome goals. Finally, research should continue to complete the reconnaissance of City department performance contracting measures and practices. Discussions with major service providers and organizations that represent them should also continue, leading to case studies of the impact of performance contracting on service delivery and to further recommendations for improved practices in the joint City-nonprofit system that typifies New York City human services. In addition, work should be done to determine the direction of New York State in establishing performance measures in its contracts and to compare these requirements with those of the City. Foundations and corporate funders are also establishing performance measures that should be included in this analysis. Jack Krauskopf is Distinguished Lecturer and Director of the Center for Nonprofit Strategy and Management at Baruch College (City University of New York), School of Public Affairs.
|