Measuring the Quality of Post-secondary Education: New Tool Points the Way
Measuring the Quality of Post-secondary Education: New Tool Points the Way
Friday, April 8, 2005 - A new paper from CPRN ventures onto contested turf to propose a new approach to help measure the quality of post-secondary education (PSE).
A number of actors - governments, media outlets, consumer organizations, and educational institutions themselves - have tried to measure the quality of education offered by Canada's universities and colleges. The results have been controversial.
The different players have competing agendas and don't agree on how to define "quality". The resulting measures are often partial and misleading, if not inaccurate. What's more, they may be less than transparent, with many not even made public.
Measuring the Quality of Post-secondary Education: Concepts, Current Practices and a Strategic Plan, by Ross Finnie, of the School of Policy Studies at Queen's University, and Alex Usher, of the Educational Policy Institute, makes progress on this front while adopting a neutral stance among the warring parties.
Finnie and Usher assess current practice in Canada and abroad, review the range of factors affecting PSE quality and outcomes, and propose a conceptual framework for improving quality measurement in future.
"We choose not to join the debate over the purpose of the PSE system," says Finnie. "We concentrate, instead, on developing an analytical approach to help identify the factors that limit or contribute to better PSE outcomes."
The Finnie/Usher framework is straightforward, capturing the PSE experience as a story of inputs and outputs whose narrative flows like this:
* Beginning Characteristics - the characteristics and abilities of incoming students that affect the quality of their educational experience and the outcomes.
* Learning Inputs - the institutional financial resources, material inputs and the organization of those resources - comprising the determinants and characteristics of individuals' learning experiences.
* Learning Outputs - the "skill sets" or any other attributes of graduates arising from their educational experiences that help determine final outcomes.
* Final Outcomes - the more specific "ultimate ends" to which the educational system may contribute - everything from employment, income and job satisfaction, to civic participation and continued education.
"This notion of quality focuses on the value-added of the educational experience," says Finnie. "The 'higher quality experiences' are those that result in superior learning outcomes, and better final outcomes."
Finnie and Usher demonstrate, with the help of their new framework, the limitations of a number of "quality measurement" exercises, from the annual Maclean's university issue to the self-evaluation exercises undertaken by institutions.
"Each of these carves out a limited part of the overall quality assessment framework. They fall short in terms of their samples, the specific data they collect, and so on," says Usher. "Many of them ignore students' beginning characteristics, for example."
The authors find significant gaps in the data sources available to flesh out their framework. Even so, they argue, enough data are available to estimate many of the empirical relationships the model suggests and advance the cause of measuring educational quality.
For the longer term, Finnie and Usher recommend further data collection to fill the gaps. The goal is a database that is longitudinal, with full sets of information on beginning characteristics, inputs, learning outcomes and final outcomes. They conclude with suggestions for how that database might be constructed, presumably under the direction of Statistics Canada.
"There is no 'silver bullet' in looking at educational quality," says Finnie, "There are no simple measures you can point to and say 'Yes, there is quality. Let's have some more of it.' But our framework does help us think about this important and difficult issue in an intelligent fashion. It will advance the cause of a better, more accountable education system more than all the smoke, mirrors, and hot-headed debates that have characterised 'the measurement of quality' to date."
Canadian Policy Research Networks Inc. - 600-250 Albert Street, Ottawa, ON K1P 6M1
Tel: (613) 567-7500 - Fax: (613) 567-7640 - Web site: www.cprn.org
courtesy of
Teresa Healy
Friday, April 8, 2005 - A new paper from CPRN ventures onto contested turf to propose a new approach to help measure the quality of post-secondary education (PSE).
A number of actors - governments, media outlets, consumer organizations, and educational institutions themselves - have tried to measure the quality of education offered by Canada's universities and colleges. The results have been controversial.
The different players have competing agendas and don't agree on how to define "quality". The resulting measures are often partial and misleading, if not inaccurate. What's more, they may be less than transparent, with many not even made public.
Measuring the Quality of Post-secondary Education: Concepts, Current Practices and a Strategic Plan, by Ross Finnie, of the School of Policy Studies at Queen's University, and Alex Usher, of the Educational Policy Institute, makes progress on this front while adopting a neutral stance among the warring parties.
Finnie and Usher assess current practice in Canada and abroad, review the range of factors affecting PSE quality and outcomes, and propose a conceptual framework for improving quality measurement in future.
"We choose not to join the debate over the purpose of the PSE system," says Finnie. "We concentrate, instead, on developing an analytical approach to help identify the factors that limit or contribute to better PSE outcomes."
The Finnie/Usher framework is straightforward, capturing the PSE experience as a story of inputs and outputs whose narrative flows like this:
* Beginning Characteristics - the characteristics and abilities of incoming students that affect the quality of their educational experience and the outcomes.
* Learning Inputs - the institutional financial resources, material inputs and the organization of those resources - comprising the determinants and characteristics of individuals' learning experiences.
* Learning Outputs - the "skill sets" or any other attributes of graduates arising from their educational experiences that help determine final outcomes.
* Final Outcomes - the more specific "ultimate ends" to which the educational system may contribute - everything from employment, income and job satisfaction, to civic participation and continued education.
"This notion of quality focuses on the value-added of the educational experience," says Finnie. "The 'higher quality experiences' are those that result in superior learning outcomes, and better final outcomes."
Finnie and Usher demonstrate, with the help of their new framework, the limitations of a number of "quality measurement" exercises, from the annual Maclean's university issue to the self-evaluation exercises undertaken by institutions.
"Each of these carves out a limited part of the overall quality assessment framework. They fall short in terms of their samples, the specific data they collect, and so on," says Usher. "Many of them ignore students' beginning characteristics, for example."
The authors find significant gaps in the data sources available to flesh out their framework. Even so, they argue, enough data are available to estimate many of the empirical relationships the model suggests and advance the cause of measuring educational quality.
For the longer term, Finnie and Usher recommend further data collection to fill the gaps. The goal is a database that is longitudinal, with full sets of information on beginning characteristics, inputs, learning outcomes and final outcomes. They conclude with suggestions for how that database might be constructed, presumably under the direction of Statistics Canada.
"There is no 'silver bullet' in looking at educational quality," says Finnie, "There are no simple measures you can point to and say 'Yes, there is quality. Let's have some more of it.' But our framework does help us think about this important and difficult issue in an intelligent fashion. It will advance the cause of a better, more accountable education system more than all the smoke, mirrors, and hot-headed debates that have characterised 'the measurement of quality' to date."
Canadian Policy Research Networks Inc. - 600-250 Albert Street, Ottawa, ON K1P 6M1
Tel: (613) 567-7500 - Fax: (613) 567-7640 - Web site: www.cprn.org
courtesy of
Teresa Healy
posted by Ilpo @ Thursday, April 14, 2005
0 Comments:
Post a Comment
<< Home