View / Download pdf version of this article

A brief history of the Programme

The PHO Performance Management Programme, later renamed the PHO Performance Programme (PPP), was established in 2005 as a voluntary programme, in response to reports from the Clinical Performance Indicator Project (2005) and Referred Services Group (2002). The main aim of the Programme is to improve the health of enrolled populations and reduce inequalities in health outcomes through supporting clinical governance and rewarding quality improvement within PHOs.

The Programme has had a number of successes in its six years. Key highlights include:

  • Improved performance against targeted indicators
  • Reduced health inequalities
  • Established strong governance arrangements including provider representation
  • Evidence of the effectiveness of primary care
  • Improved data for quality improvement, policy development and planning
  • Alignment with other performance measures to minimise the collection burden on primary care

These successes have been achieved through hard work and good sector engagement, but this was not always an easy path.

The initial set of Programme indicators were developed by a PHO Clinical Performance Advisory Group and a joint DHB/Ministry of Health project team, with guidance from a Referred Services Management Expert Steering Group. The first pre-requisite phase began in July 2005 and the first six monthly performance periods began on 1 January 2006, with 29 PHOs participating. Despite some initial criticisms, participation in the Programme continued to grow and by July 2007 all of the then 82 PHOs had joined.

A new governance structure for the PPP was introduced in 2008, involving provider, PHO, DHB and Ministry of Health representatives. This Governance Group still exists today and has provided strong leadership and guidance for the Programme to ensure it continues to add significant value to the sector. The Governance Group is supported by the Programme Advisory Group, which provides expert advice on the content of the Programme and ensures clinical relevance and evidence base as well as considering implementation options, policy priorities and alignment and business sustainability.

Under these governance arrangements the Programme has matured and it now places a greater emphasis on quality improvement. This is achieved via a step-wise approach, starting with recording, then moving on to activities and finally on to patient outcomes. An example of this is the management of patients with diabetes. Measures currently exist to record a diagnosis of diabetes and to actively manage patients who have been detected through regular reviews. Now a third phase measurement is being considered to assess how well these patients are responding to their management. Once recording is well established, it is planned that these indicators will cease to be funded directly, but will transition to qualifying criteria for payment of incentives for the management of patients with diabetes.

The continuous improvement approach is supported by a number of incentive mechanisms. Incentive payments are distributed based on the rate of progress being made rather that just absolute performance against the target agreed between the PHO and the DHB. This mechanism recognises that PHOs serve diverse populations with different needs and a one size fits all approach, based on nationally consistent targets, could lead to the undesirable situation where PHOs serving the highest need communities with the most difficult to reach populations would receive the smallest incentive payments.

The reduction of health inequalities also remains an important objective. This is recognised in the incentive payments structure where a higher payment is given to PHOs for successful improvement in the higher need population. For the purpose of the Programme, the high need population is defined as; people who are of Māori or Pacific ethnicity or who live in the most deprived quintile areas.

Other incentive mechanisms include the development of business intelligence tools for the PHOs and the distribution of information in ways that are useful for PHOs (and their practices) and DHBs to inform improvement.

Has the Programme delivered value?

Earlier this year the Programme surveyed its stakeholders to assess its value. Whilst this survey only received 90 responses these were well distributed among Providers, PHOs and DHBs. Figure 1 illustrates that overall, most respondents found the Programme to be valuable.

Figure 1: PPP stakeholder survey 2011; summary of key results


How has the Programme measured up?

The Programme has now been running for approximately six years and for many Programme indicators there is sufficient history to identify whether improvements have been made in the target populations.

Whilst it could be argued that some of the improvements reported are merely down to better recording of activity that was already being undertaken in General Practice before the Programme was established, it is also true that systematic recording and measurement is an element of a quality system.

“We can only be sure to improve what we can actually measure”.
– Lord Darzi, Next Stage Review report, NHS, 2008

As with any other complex system it is sometimes difficult to isolate the success of the Programme from other factors influencing performance.

With these caveats the performance against some of the key indicators covering health priority areas are examined as follows.

Age appropriate vaccination for two-year-olds

This indicator was originally measured using data derived directly from primary care and counted the proportion of enrolled two-year-old children recorded as being fully immunised. Following discussions with PHOs and extensive patient-by-patient analyses proving the reliability of the national immunisation register (NIR), this indicator was transitioned to using the NIR as the data source by 1 January 2011. Now this indicator measures the percentage of children aged two years who have received the complete set of final dose age appropriate vaccinations. The number of fully vaccinated children is compared with the number of children on the enrolment register who turned age two years during the period, even if they are not recorded on the NIR. For this reason the Programme usually reports slightly lower rates of immunisations than are quoted for DHBs in the Health Target where only children recorded on the NIR are reported.

The Programme goal was originally set at 85% but has recently been raised to 90%, and will further increase to 95% in 2012, to maintain alignment with the Health Target.

Since this indicator was first funded in October 2007 PHOs have continued to improve their performance. The current rate of immunisation in two-year-olds is 87.2% amongst the high need population and 88.0% for the total population (Figure 2). The gap between the rates achieved for the high need population and the total population continues to close and is now just 0.8%.

Figure 2: Percentage of children aged two years, enrolled in a general practice in New Zealand, who have been fully immunised
Childhood immunisation - 2 Year Olds

Breast cancer screening

Originally this indicator measured the percentage of women aged 50 to 64 years who had a mammogram as part of the national breast screening programme (BreastScreen Aotearoa), within the last two years. In January 2011 this age range was extended to include women aged 65 to 69 years to reflect the current target population for the national breast screening programme. This change has increased the reported figures for this indicator by 0.4% in the latest reporting period.

Only publicly funded mammography screenings performed by BreastScreen Aotearoa are counted by the Programme. True coverage rates, which include private mammography screenings, are likely to be higher, particularly for the total population measure, which is an information only indicator.

Breast cancer screening rates have improved steadily over the time period, with a 13% rise in rates recorded overall and a 19% improvement in rates for the high need population – reducing reported health inequalities (Figure 3). Provided current rates of improvement are maintained, the Programme goal of 70% coverage should be achieved for the high need population group within the next two years.

Figure 3: Percentage of women aged 50 to 64 years (50 to 69 years from 2011), enrolled in general practice in New Zealand, who have been screened by the national breast screening programme within the last two years
Breast cancer screening

Cervical cancer screening

This indicator measures the percentage of women aged 20 to 69 years who have received a cervical screen within the past three years. Data provided by the National Cervical Screening Programme is used to calculate this measure so women who choose to opt off the National Screening Programme’s register will not be counted by the Programme, even if they have had a cervical screen within the past three years. All figures are also adjusted at the national age related rate to allow for women who have had a hysterectomy and do not require cervical screening.

Although progress has been made over the lifetime of the Programme and rates have risen by 5%, improvement has now slowed and the national rate remains just below the Programme goal of 75% coverage (Figure 4). Progress has been slightly better for the high need population and the gap between the two populations has now reduced to 8% from over 11%.

Figure 4: Percentage of women aged 20 to 69 years, enrolled in general practice in New Zealand, who have had a cervical screening test within the last three years
Cervical cancer screening

Ischaemic cardiovascular disease detection

This indicator measures ischaemic CVD diagnoses recorded in primary care against a national model of expected rates for patients aged 30 to 79 years.

Prior to 2010 there were problems obtaining complete data for this indicator from primary care systems and collection was limited to a subset of ischaemic heart disease. High levels of detection are now being reported, with all PHOs recording detection rates in excess of prevalence estimates (Figure 5). Work is being conducted to understand why such high rates are being reported and to ensure that, in the future, more realistic performance figures are produced by the Programme. The number of patients being recorded with ischaemic CVD in primary care has risen steadily over the past three reporting periods.

Figure 5: Percentage of people, enrolled in general practice in New Zealand, who have been diagnosed with ischaemic CVD
Ischaemic CVD detection

Cardiovascular risk assessment

This indicator measures the percentage of enrolled patients in the target population who have had their cardiovascular risk recorded within the past five years. This indicator was introduced in 2008 with a Programme goal of 80% coverage to be achieved within five years.

The target population for this indicator is:

  • Males of Māori, Pacific or Indian sub-continent ethnicity aged 35 to 74 years
  • Females of Māori, Pacific or Indian sub-continent ethnicity aged 45 to 74 years
  • Males of any other ethnicity aged 45 to 74 years
  • Females of any other ethnicity aged 55 to 74 years

Although the capture of CVD risk information has been occurring in primary care across the country for many years, this indicator is relatively new to the Programme. Improvements continue to be made in the systematic recording of this information within practice management systems and subsequent capture by the Programme. Currently the rate of CVD risk assessment is well below the 80% Programme goal (Figure 6).

Figure 6: Percentage of people in the target population, enrolled in general practice in New Zealand, who have had a CVD risk assessment within the past five years
CVD risk assessment

Diabetes Detection

The diabetes detection indicator compares the number of patients aged 15 to 79 years, recorded with a diagnosis of diabetes in primary care, against a national model of expected rates of diabetes, based on indicators of disease taken from other sources such as pharmaceuticals dispensed, laboratory tests ordered and hospital records.

Progress against this indicator is complicated by refinements made to the national diabetes prevalence model. In fact, one of the successes of the Programme has been the refinement of the national prevalence model as a result of feedback that primary care has been able to provide the diabetes team at the Ministry of Health. Despite these improvements, the prevalence model continues to understate the number of people with diabetes, particularly amongst the Māori and Pacific population. As a result, detection numbers exceed 100% (Figure 7).

Interestingly, a recent report from the National Health System (NHS) in the United Kingdom found that 2.2% of patients classified as having diabetes, did not actually have this condition. If this was also the case in New Zealand, it would provide further explanation of the current Programme results.

Figure 7: Percentage of people, enrolled in general practice in New Zealand, who have been diagnosed with diabetes
Diabetes detection

Diabetes Detection and Follow Up

This indicator measures the proportion of the population with diabetes, aged 15 to 79 years, who are receiving an appropriate diabetes review at least annually. A diabetes review is expected to include the measurement of HbA1c, microalbuminuria and lipids, review of cardiovascular risk, examination of the feet, retinal screening (every two years) and review and updating of the patient’s care plan. The Programme goal is that at least 80% of this population should receive a diabetes review each year.

Currently there are technical difficulties in collecting this data from PHOs who do not use the Get Checked Programme to provide diabetes reviews; these difficulties are being addressed by the Programme on a case by case basis.

This indicator currently uses the national prevalence model as the denominator, rather than the number of patients identified by primary care. Therefore, improved performance against this indicator partly reflects changes in the prevalence model (Figure 8). When compared against the number of people with diabetes recorded by primary care, annual review rates have risen from 51% to 65.8% (total population) and 53.6% to 65.9% (high need population) since July 2008. Some of this reported increase reflects improvements being made in collecting this data via PHOs.

Figure 8: Percentage of people with diabetes, aged 15 to 79 years, enrolled in general practice in New Zealand, who have received an appropriate diabetes review at least annually
Diabetes detection follow up

Seasonal influenza vaccination coverage

This indicator measures the rate at which patients aged over 65 years receive seasonal influenza vaccinations, each year between January and June. This data is sourced from claims data rather than directly from primary care so patients paying for their own vaccinations or where vaccinations are delivered outside a practice setting may not be counted.

Although some progress was made in the first few years of the Programme, additional progress has been slow for this indicator since then and the coverage rate reported by the Programme remains 10% below the Programme Goal of 75% coverage (Figure 9).

Figure 9: Percentage of people aged over 65 years, enrolled in general practice in New Zealand, who have received a seasonal influenza vaccination
Flu vaccination coverage

Smoking Status Recorded

This indicator measures the proportion of the enrolled population aged 15 to 74 years who have had their smoking status recorded. The Programme goal is that smoking status will be recorded for at least 90% of the population. A minimum of 70% of this population must have their smoking status recorded before the PHO is eligible to receive incentive funding for primary care to provide brief advice or cessation support. This applies to the performance period ending 31st December 2011 and in subsequent years.

This indicator is measured from data taken from practice management systems and provided to the Programme as a PHO aggregate.

This indicator was only introduced as a funded indicator from 1 January 2011 so it is too early to assess any significant trends. Overall the results show an encouraging positive trend in performance, which has been most rapid for the high need population group, typically resulting in more equitable performance between the population groups (Figure 10).

Figure 10: Percentage of people, aged 15 to 74 years, enrolled in general practice in New Zealand, who have had their smoking status recorded
Smoking status recorded

Programme successes

The Programme has been instrumental in building a rich database of evidence of the work being done in primary health, to improve the health of the population through immunisation, screening, health promotion and the management of longer-term conditions. This is invaluable in helping inform policy and planning decisions around the future of health care.

The Programme has helped develop standards for the systematic recording of specific health information, and where necessary, funding changes to practice management systems. This has led to improved data quality - debates about the quality of the data are now more often about what the information is saying and how this can be used to inform better quality care. The sector is now placing increasing demands on the Programme to deliver more detailed timely data.

The Programme is currently working with the Ministry of Health and other agencies to ensure, wherever possible, that Programme indicators align with similar measures used elsewhere and, where practical, ensure that data collected by primary care providers form the basis for primary care health targets.

The Programme also funds educational material for primary care including contributing to the funding of Best Practice Journal and other bpacnz material.

Development plans

The Programme has been successful in establishing robust data flows for the recording of activity for health priority areas. Further work is now required to evolve the indicators in these areas to focus more on health outcomes and the benefits to patients and the community.

Some important areas of primary care activity need to be added to the Programme to provide a more diverse range of measures. For example, work is currently underway to consider suitable indicators to measure and promote good primary mental health care.

One of the criticisms of the Programme has been that the information provided is produced far too late and too remote from primary care providers to be useful in changing behaviours and assisting effective management. This is currently a major focus for the Programme team and reports are now being produced more promptly. Monthly reports are being created for indicators, where the report can be produced without adding to the collection burden of PHOs and providers. This allows progress to be tracked and actions taken within each reporting period.

The feasibility of providing prospective target lists, available at practice level for screening and immunisation activity, is being considered. This would enable individual practices to readily identify the work they need to do to achieve targets.

Data quality remains a concern and recent validation work undertaken by the Programme has revealed a small number of issues with PMS vendor implementations of the data extracts used by the Programme. Local variations in the means of delivering or coding a service also continue to present problems for the Programme and these will continue to be addressed on a case-by-case basis as the Programme becomes aware of them.

The merger of PHOs has created a number of challenges for the Programme with some of the new, larger organisations wishing to report performance at a locality level. This is relatively simple for indicators reported from national collections, such as immunisation and screening activity, that are available at patient level. However, the task is far more difficult for indicators derived from practice management systems since the Programme has traditionally only received PHO level aggregates to meet the concerns around patient privacy. Local arrangements are being made with these PHOs to receive more detailed information that can then be reported at a locality level. The Programme is proposing to strengthen its processes to ensure that this can be achieved without compromising patient privacy.

Patient level information has also proved useful in the past, in helping to improve the quality of the data received by the Programme. This provides a useful audit trail that is currently absent from the PHO aggregate data received by the Programme.

The other big challenge to the Programme will be to embrace the multidisciplinary team approach to patient care and ensure that it does not create new blind spots in the collection of data. Perhaps General Practitioners, as co-ordinators of care for their patients, will continue to remain the best place to seek this information, but the Programme will need to remain alert.

Final thoughts

The Programme has developed, and maintains, a rich source of data describing primary care activity and the health of the enrolled population. This is useful to policy makers, planners and funders and provides comparative data that can be used by PHOs and providers to identify opportunities for quality improvement. Overall, progress in recorded performance has been made across all currently funded indicators since they were first introduced to the Programme, although some indicators create significant challenges if the Programme Goals are to be achieved. Perhaps most importantly the Programme has raised the visibility of the activities of primary care in helping to keep New Zealanders healthy and narrowing the gap between high need populations and the rest of the country.

For further information about the PHO Performance Programme, including membership of the Advisory and Governance groups, visit:

ACKNOWLEDGEMENT Thank you to Stewart Pye, Senior Analyst, National Services and Primary Health Performance, DHB Shared Services and colleagues for contributing this article.