Tautoko ā-pāremata

Ngā pepa rangahau

Content provider
Parliamentary Library
Information
Date:
26 April 2012
Downloads
Note: The above document(s) are provided as an Adobe PDF (PortableDocument Format) file. you can download a free viewer for PDF files from Adobe's web site.
Contact details
Parliamentary Library
T: 64 4 817 9647
Email contacts

Performance-Based Research Fund

April 2012

Introduction

The Performance-Based Research Fund (PBRF) is administered by the Tertiary Education Commission (TEC). The size of the fund is $250 million a year. This is just under 9% of the approximately $2.8 billion a year allocated by the TEC on behalf of the Government to fund tertiary education provision. [1]  

The function of the PBRF is to encourage tertiary education providers to produce high-quality research. This is achieved by regular assessment of the research performance of eligible providers. All New Zealand-based degree-granting tertiary education providers (and subsidiaries that are wholly-owned by such providers) are eligible to participate in the PBRF.

The PBRF is a fixed pool of money distributed amongst all eligible providers according to a formula based on assessed performance. The formula-based model of funding distribution ensures that the entire fund is allocated each year. It also means that if one provider performs exceptionally well, other providers must receive a proportionally smaller amount of money from the pool.

Although any tertiary education provider with degree-granting authority may participate in the PBRF, universities receive by far the greatest proportion of the fund (around 97.3%). [2]  

The PBRF allocates regular bulk funding to tertiary education providers on the basis of their performance on the three measures described below. Because it is bulk funding providers may spend their PBRF allocation as they deem fit. The PBRF is not a mechanism by which the Government commissions or purchases individual research projects.

Origins of the PBRF

The PBRF was initiated as one of a series of measures intended to move tertiary education funding away from the model used in the 1990s, whereby funding for research was linked to numbers of student enrolments. Beginning in 1999, and culminating with the publication in 2002 of the Report of the PBRF Working Group, a funding mechanism was developed that would directly connect funding with measured research quality.

The Parliamentary Library has published a 2004 Background Note with more detailed information on the origins of the PBRF.

Components of the PBRF

Performance assessment is undertaken using three separate weighted measures. These measures are:

  • The measurement of External Research Income (ERI), weighted at 15% (around $37.5m);
  • The measurement of Research Degree Completions (RDC), weighted at 25% (around $62.5m);
  • A periodic Quality Evaluation, weighted at 60% (around $150m).

The ERI and RDC components of PBRF are measured annually. The Quality Evaluation is intended to be undertaken every six years. Since the PBRF was implemented in 2002 there have been two Quality Evaluations. The first was in 2003, the second was a partial round undertaken in 2006. The next Quality Evaluation takes place in 2012.

Eligible providers must participate in the Quality Evaluation in order to receive ERI and RDC funding. This fact, along with its high weighting, makes the Quality Evaluation the most significant element of the PBRF. The main elements of the Quality Evaluation are set out in the remainder of this paper.

Design of the 2012 Quality Evaluation

Throughout 2008 and 2009 there was a period of consultation with the tertiary education sector about the design of the Quality Evaluation. This consultation was overseen by an independent Sector Reference Group and consisted of a series of consultation papers on a variety of aspects of the Quality Evaluation. Consideration of the response to these consultation papers resulted in the Sector Reference Group making a set of 87 recommendations to the TEC on the design of the 2012 Quality Evaluation.

As in the previous two Quality Evaluations, the 2012 Quality Evaluation will seek to measure the quality of research produced at an institution by evaluating the research of all individual eligible staff members at that institution and aggregating the results. All staff engaged in research will be required to produce an Evidence Portfolio (EP). The EP provides an overview of research and other relevant activities during the assessment period. The staff member also selects up to four of their research outputs as Nominated Research Outputs to be made electronically available for assessment.

Quality Evaluation process

Assessment of research quality is made by independent peer review panels. There are twelve of these panels, each covering a separate academic field. Each panel has a Chair and Deputy Chair and is comprised of around a dozen to twenty recognised experts. The TEC has published a complete list of all panel members. In addition there is a moderation panel that oversees the peer review panels to ensure consistency.

Peer review panels and Expert advisory groups

Providers select by which of the twelve peer review panels each of their staff member’s EPs will be assessed. The final allocation is made by the TEC in consultation with panel Chairs. Some EPs may be cross-referred from one panel to another.

The twelve peer review panels are as follows.

In addition to the peer review panels there are also two Expert advisory groups. If a provider decides that an EP contains material within the ambit of one of these specialist groups then that EP will be cross-referred to the group to allow the group to provide input to the panel assessment of the EP.

The Expert advisory groups are as follows.

Assessment

Each individual staff member’s EP goes through an extensive assessment procedure beginning with an initial assessment by two panel members. The full panel then considers each EP, assigning a series of interim scores that are refined through a sequence of steps until a final Quality Category is arrived at. This procedure takes regard of a number of relevant aspects of the staff member’s research, including the special circumstances of that individual. There is also scope for an EP to be cross-referred to another subject panel and for a specialist advisor to be consulted if necessary. [3]  

New or emerging researchers

In recognition of the fact that it takes some time to build up a platform of research, those who have begun their research careers on or after 1 January 2006 are designated new or emerging researchers and may be assigned one of two special Quality Categories.

Quality Categories

There are six Quality Categories. Two of them are only available to be assigned to new or emerging researchers. The Quality Category descriptors [4]   are:

Quality Category “A”: For an EP to be assigned an “A” it would normally be expected that the staff member has produced research outputs of a world-class standard, established a high level of peer recognition and esteem within the relevant subject area of their research, and made a significant contribution to the New Zealand and/or international research environments.

Quality Category “B”: For an EP to be assigned a “B” it would normally be expected that the staff member has produced research outputs of a high quality, acquired recognition by peers for their research at least at a national level, and made a contribution to the research environment beyond their institution and/or a significant contribution within their institution.

Quality Category “C”: For an EP to be assigned a “C” it would normally be expected that the staff member has produced a reasonable quantity of quality-assured research outputs, acquired some peer recognition for their research, and made a contribution to the research environment within their institution. This Quality Category is available for the EPs of all PBRF-eligible staff members except new and emerging researchers.

Quality Category “C(NE)”: For an EP to be assigned a “C(NE)” a new or emerging researcher would normally be expected to have produced a reasonable platform of research, as evidenced by having either a) completed their doctorate or equivalent qualification and produced at least two quality-assured research outputs, or b) produced research outputs equivalent to a doctorate and at least two quality-assured research outputs. This Quality Category is available for the EPs of new and emerging researchers only.

Quality Category “R”: An EP will be assigned an “R” when it does not demonstrate the quality standard required for a “C” Quality Category or higher. This Quality Category is available for the EPs of all PBRF-eligible staff members except new and emerging researchers.

Quality Category “R(NE)”: An EP will be assigned an “R(NE)” when it does not demonstrate the quality standard required for a “C(NE)” Quality Category or higher. This Quality Category is available for the EPs of new and emerging researchers only.

Quality Evaluation timeline

The PBRF Census date is 14 June 2012. This date is the reference point for determining the eligibility both of providers and also staff members. Tertiary providers wishing to participate in the PBRF must have degree-granting authority on this date. Staff member’s employment status in the year preceding the census date is a major factor in determining their eligibility. [5]  

The panel assessment of EPs takes place at the end of November 2012.

The results of the Quality Evaluation are expected to be available in April 2013.

The TEC website has a detailed timeline of the Quality Evaluation process.

Further reading

Along with the material already referred to above and in footnotes, the following works are recommended for those who wish to learn more about the PBRF.

The TEC has published comprehensive reports on the results of the 2006 and 2003 Quality Evaluations.

The Ministry of Education has published analysis comparing university tuition fees with PBRF performance and examining trends in the commercialisation of university research.

Dr Damien Cole

Research Services Analyst

Parliamentary Library

Email: Damien.Cole@parliament.govt.nz

This work is licensed under the Creative Commons Attribution 3.0 New Zealand licence. In essence, you are free to copy, distribute and adapt the work, as long as you attribute the work to the Parliamentary Library and abide by the other licence terms. To view a copy of this licence, visit http://creativecommons.org/licenses/by/3.0/nz/.

  1. Tertiary Education Commission, Annual Report 2011, pages 8, 58.   [back]
  2. Tertiary Education Commission, PBRF Annual Report 2009, page 13.   [back]
  3. For a detailed description of the assessment process please see Chapter 3 of Tertiary Education Commission, PBRF Guidelines 2012.   [back]
  4. As stated in the PBRF Guidelines 2012, page 109.   [back]
  5. For more details on how staff eligibility is determined see Chapter 2 Section B of PBRF Guidelines 2012.   [back]